Metaphor Detection by Deep

Learning and the Place of Poetic

Metaphor in Digital
Humanities

Chris Tanasescu (Margento)

Vaibhav Kesarwani
Diana Inkpen
The Graph Poem Project--Applying Graph
Theory in Poetry
• the Graph Poem project started as a creative writing and
generative initiative, writing, assembling, (post)digitally and/or
algorithmically generating and/or expanding poetry corpora
based on commonalities between poems.

• Poems are nodes in a network graph while edges represent

commonalities for which computational tools have been
developed (classifiers for poetic features converging into a
holistic approach).

• So far we have developed topic, meter, rhyme, diction, and

(ML/statistical) metaphor classifiers.
The Graph Poem in DH

• We combine critical and analytical with creative and generative

purposes; thus joining a more general trend in digital humanities
(DH)—in what was called the second and third wave in DH
(Berry 2011)—a transition from classification or retrieval to more
creative approaches and tools, or moreover, a fusion of the two.

• Adopting a “making; critique” (Hayles and Pressman 2013)

approach.

• The issue of data in literature and poetry; from big data to data-
intensive approaches.
The Quantitative-Qualitative Components. DH &
Deep Learning
• In the existing work it is true that the magnitude of the explored
data is relative and not necessarily consequential in terms of
quality of results;

• This aspect becomes of the essence though in developing deep

learning (DL) approaches, as is the case of the work presented
in this paper.

• As will see below, DL proves to provide better results than other

methods, but in order for this method to work, the amount of
data has to be increased beyond certain limits.
Graph Theory Apps Background

• Graph theory has never been deployed in poetry before, but

network graphs have been used in other literature-focused
chapters of DH (fiction) and computational linguistics.

• Differences between these applications and the ones pertaining

to the Graph Poem project:

• 1. genre-related;

• 2. all of the graphs in fiction analysis are plain (single-layer)

networks whereas the ones assembled and analyzed in the
latter are multiplex networks (or multigraphs).
Metaphor Detection—General Context and
Previous Graph Poem Work
• The most elaborate metaphor-focused DH project to this day is
the Metaphor Map of English at the University of Glasgow
(Hamilton, Bramwell, and Hough 2016) that visualizes the use of
metaphor in the Historical Thesaurus with links between various
categories. It does not work as an automatic classifier, and it is
limited to the Thesaurus, with no option to run the tool on any
other data.
• The only DH project that has aimed to process poetic metaphor
computationally is POEMAGE (Coles 2017). Unlike the Graph
Poem, though, POEMAGE does not involve creating a tool to
algorithmically identify and visualize metaphor (Coles 2017) but
“an oblique way” of pointing the human reader to places in the
text the machine is uncertain of in terms of pronunciation and
meaning and thus signalling potential metaphor occurrences.
Metaphor Detection—General Context and
Previous Graph Poem Work (II)
• Graph Poem metaphor classifier (Kesarwani et al. 2017): a rule-
based method, and a statistical model.
• We developed our own corpus and did manual annotation: 720
sentences out of 1500 extracted, with two annotators and an
arbiter deciding on disagreements (PoFo). We combined our
dataset with two other sets, TroFi (Birke and Sarkar 2006) and
Shutova (Mohammad et al 2016); The results for PoFo + TroFi +
Shutova were significantly better than PoFo alone.
TWOFOLD CONCLUSION:
• A. In detecting metaphor in poetry, non-poetry data are as
helpful as poetry data.
• B. Tools trained on poetry data are sometimes potentially better
for non-poetry tasks than the ones trained on non-poetry data.
Generic Metaphor Detection Web-based App
Generic Metaphor Detection Web-based App
(II)
• We used Stanford NLP parser dependencies to extract the
potential word pairs:
• nsubj
• dobj
• nsubjpass
• acl:relcl

• The application accepts multi-line text and outputs line-by-line

result for the analysis. There can be multiple word pairs for each
line.
Generic Metaphor Detection Web-based App
(III)
• Poem Title : Afterimages (excerpt)
• Author : Audre Lorde

A woman measures her life’s damage

my eyes are caves, chunks of etched rock
tied to the ghost of a black boy
whistling
crying and frightened
her tow-headed children cluster
like little mirrors of despair
their father’s hands upon them
and soundlessly
a woman begins to weep.
Generic Metaphor Detection Web-based App
(IV)
• Line : A woman measures her life’s damage Processing measures : woman
Prediction : metaphor
• Processing measures : damage Prediction : metaphor
• Line : my eyes are caves, chunks of etched rock Processing caves : eyes
Prediction : metaphor
• Line : her tow-headed children cluster
• Processing cluster : children
• Prediction : metaphor
• Line : like little mirrors of despair
• Processing like : mirrors
• Prediction : metaphor
• Line : their father’s hands upon them
• Processing hands : father
• Prediction : literal
• Line : a woman begins to weep.
• Processing begins : woman
• Prediction : literal
 Convolutional neural networks (CNN)

In CNNs, we use convolutions over the input layer to compute the

output. This results in local connections, where each region of the input
is connected to a neuron in the output. Each layer applies different
filters, typically hundreds or thousands, and combines their results.
Deep Learning Metaphor Classification

• The first requirement for DL classification is lots of examples /

data points (by the thousands). We collected significant (poetry
and non-poetry) metaphor data: around 4870 instances.

• We experimented with Convolutional Neural Networks (CNN)

(Kim 2014) to examine whether we can get any gains in F-score
when compared to the standard machine learning classifiers.

• We used the Keras (Chollet et al. 2015) DL framework with a

Tensorflow (Abadi et al. 2015) backend and used a local GPU to
accelerate the training process.
Range of parameters tested
CNN Classification
Discussion of Results

• The best result, i.e., F-score 0.833 for metaphor and F-score
0.744 for the non-metaphor class was seen with epochs 300,
batch 70, neurons 206 and inputs 103.
• It can be observed that with the same training and test set, CNN
performed significantly better than SVM and other machine
learning algorithms. The best F-score for metaphor class was
0.781, seen with SVM with Puk kernel. For CNN, we get a gain
of 5.2% and we get a high F-score of 0.833.
• For the non-metaphor class, KNN obtained a F-score of 0.711.
For CNN, we get a gain of 3.3% and we get a higher F-score of
0.744.
• For both classes, the performance is better with CNN.
Discussion of Results (II)

• The major drawback for the CNN classification (when compared

to the other ML algorithms) is that a lot of data points are
needed for training.
• Consequently, though we got better results for
PoFo+TroFi+Shutova collectively, results on the rest of the
datasets individually were worse than the SVM results for both
the classes.
• We empirically observed that, in our case, anything less than
3,000 instances was insufficient for training the CNN and the
results are much worse (close to the baseline of 50%).
• If we are able to overcome this soft threshold of 3,000 data
points, deep learning classification works appreciably better.
Conclusion and Future Work
• The Graph Poem is the first initiative in assembling a
comprehensive holistic conglomerate of poetry algorithms and
tools.
• For style, so far we have developed metaphor classifiers that
deploy rule-based, statistical (machine learning), and deep
learning methods, and we have launched a web-based
metaphor natural language processing (NLP) application.
• We verified the validity of the data intensive approach: we have
established that DL, and particularly CNNs output better results
than the other previously deployed methods, given that the
amount of data is big enough.
• Future work: a generic metaphor classifier not dependent on
syntactical patterns; integrating the resulting web-based
application into the overall graph analysis and visualization tool.
Thank You 

• Questions, comments?

• http://artsites.uottawa.ca/margento/en/the-graph-poem/
(The Graph Poem webpage @ uOttawa)

• http://bit.ly/2KDIY6F
(Kesarwani’s thesis)

Contact: ctanase2@uottawa.ca
Diana.Inkpen@uottawa.ca