Distant Listening to Gertrude

Stein’s ‘Melanctha’: Using

Similarity Analysis in a Discovery
Paradigm to Analyze Prosody
and Author Influence
............................................................................................................................................................

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

.......................................................................................................................................
Correspondence:
Tanya Clement, School of
Information, University of
Texas at Austin, Austin.
Email:
tclement@ischool.utexas.edu
Tanya Clement
School of Information, University of Texas at Austin, Austin
David Tcheng, Loretta Auvil and Boris Capitanu
Illinois Informatics Institute, University of Illinois at Urbana-
Champaign, Urbana
Joao Barbosa
Texas Advanced Computing Center, University of Texas at Austin,
Austin
Abstract
Used here to describe the investigation of significant sound or prosodic patterns
within the context of a system that can translate these patterns into comparative
visualizations across texts, the term ‘distant listening’ is used provocatively to
suggest that readers might interpret prosodic patterns as ‘noise’ (or seemingly
unintelligible information) with close reading practices. In this study, we show
that these same patterns appear coherent and discoverable within ProseVis, a
visualization tool that supports these hermeneutics within a discovery-based
paradigm that allows for new ways of making meaning. Charles Bernstein dis-
cusses ‘close listening’ as possibly contradictory to ‘ ‘‘readings’’ of poems that are
based exclusively on the printed text and that ignore the poet’s own perform-
ances, the ‘‘total’’ sound of the work, and the relation of sound to semantics’
(Bernstein, 1998, p. 4). Likewise, this study considers the efficacy of using pros-
odic textual elements as features for similarity metrics instead of or alongside
words and n-gram frequencies. In particular, this discussion describes the con-
tinued development of this work as a contribution to and within the context of
authorship attribution and stylometric studies that consider the interpretability
of prosodic features. To that end, in the first part of this discussion, we place the
study within the theoretical and practical context of author attribution studies. In
the second part of this discussion, we consider how changing similarity metric
calculations through the inclusion and exclusion of certain prosodic features
(such as tone and stress) and algorithmic parameters (such as the window size
of sounds and weighting power) can facilitate the discovery of previously

Literary and Linguistic Computing, Vol. 28, No. 4, 2013. ß The Author 2013. Published by Oxford University Press on 582
behalf of ALLC. All rights reserved. For Permissions, please email: journals.permissions@oup.com
doi:10.1093/llc/fqt040 Advance Access published on 23 July 2013
 Distant listening to Gertrude Stein’s ‘Melanctha’

unidentifiable author-similarity patterns. Finally, in the third part of this study,

we explore questions of identity construction within this framework of author
attribution analysis by comparing ‘Melanctha’, the longest story in Gertrude
Stein’s Three Lives (1909), with 150 different narrative voices from the First
Person Narratives of the Documenting the American South collection.
.................................................................................................................................................................................

1 Using Prosody Features in Indeed, Efstatios Stamatatos calls the use of topic-
independent words such as function words, the
Similarity Metric and Author ‘pure stylistic choices of the authors across different
Attribution Studies topics’ (2009, p. 540). Burrows describes author at-
tribution studies that use ‘weak discriminators’ such

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

Traditional author attribution studies are often per-
formed to determine the singular author of a given
text. Some of the most often cited examples include
determining the author or authors of the Federalist
Papers (Forsyth and Holmes, 1996; Jockers and
Witten, 2010; Mosteller and Wallace, 1964) and
John Burrows’ work using function words to deter-
mine Jane Austen’s particular style (1987) and to
determine the authorship of English Restoration-
era poets (2002). Often these studies are profile-
or instance-based, ‘closed games’ (Burrows, 2002,
p. 267) in which the texts are of equal length, the
corpora only includes a handful of mostly known
authors, the language of the texts are normally ‘in
the most similar register’ (Grieve, 2007, p. 255) or of
equivalent dialects, and the texts pertain to ‘a similar
range of topics’ (p. 256). For the most part, these
studies have successfully examined the extent to
which a variety of measures, different features of
study, and different parameters and algorithms
achieve a variety of better or worse results.
Most of these author attribution studies focus
primarily on the use of high-frequency words or
n-grams to establish similarities among authorial
styles (Burrows, 2002; Diederich et al., 2000;
Grieve, 2007; Hoover, 2003a,b; Juola et al., 2006;
Koppel et al., 2007; Martindale and McKenzie,
1995; Uzuner and Katz, 2005; Yu, 2008; Zhao and
Zobel, 2005). High-frequency function words are
ordinarily chosen because classification algorithms
are sensitive to similarities among low-frequency
content (Grieve, 2007, p. 260) or ‘context-specific’
words (Jockers and Witten, 2010, p. 217). Although
using any word-based metrics might seem to cinch
an analysis of style to a study of content or topic, the
authors cite function words to be context free.
as these high-frequency function words to deter-
mine an author’s ‘stylistic signature’ as the study
of ‘tiny strokes’ (2002, p. 268).
Studies use high-frequency function words as
features for studying style and attributing author-
ship for quite valid reasons concerning efficiency
and interpretable results, but it is misleading to sug-
gest that high-frequency words have proven to be
the most productive or accurate textual features for
studying style. Jockers and Witten (2010), who
focus on performing a benchmarking study on
determining the best classifier for authorship attri-
bution problems, use high-frequency words and
n-grams with little regard for studies that use
other features even as they cite feature selection
as one of the most significant factors in machine
learning classification techniques (2010, p. 215).
To the contrary, Grieve’s study, among others
(Forsyth and Holmes, 1996; Sanderson and
Guenter, 2006; Zhang and Lee, 2006), proves that
using graphemes as a feature yields very favorable
results, commenting that ‘they are the most frequent
potential indicator of authorship in any English text,
and as such any patterns in their usage will have a
better chance to emerge’ (2007, p. 261). Sanderson
and Guenter (2006) also test using character n-
grams of variable length as features with short
English texts and produce the best results when
examining character sequences of up to 4-grams.
Forsyth and Holmes (1996) likewise find that
using character n-grams yields better results than
lexical features in many text-classification tasks,
including authorship attribution. So too, in Juola’s
competition to prove the best performing author
attribution algorithm, one of the best performers

Literary and Linguistic Computing, Vol. 28, No. 4, 2013 583

T. Clement et al.

uses character n-grams as features (Juola, 2004; something goes awry, we may have difficulty
Juola et al., 2006). in attributing this to problems with the col-
Most of these studies do not use syntactical fea- lection process or the specification of the
tures such as part of speech or sentence and phrase features (Weiss et al., 2005, p. 51).
structure, primarily because syntactic parsers
On the other hand, Weiss et al. maintain (like
produce many errors and therefore noise. On the
Burrows and others) that the results of text
other hand, Stamatatos cites several studies
mining procedures are easier for developers than
(Baayen et al., 1996; Gamon, 2004; Stamatatos
more quantitative data mining, because the results
et al., 2000, 2001) in which ‘results have shown
include whole words. ‘For text mining’, they write,
that this type of measure performs better than do
‘we are much closer to understanding the data, and
vocabulary richness and lexical measures’ (2009,
we all have some expertise. The document is text.
p. 542). At the same time, Baayen et al. note that

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

We can read and comprehend it, and we analyze a
the use of function words is an ‘economical’ way to
result by going directly to the documents of interest’
measure syntactic features because these words are
(emphasis added; Weiss et al., 2005, pp. 51–2).
tied to syntactical patterns, but they posit that
Certainly, using textual features other than the
syntax-based methods are more robust and lead to
word, such as graphemes and syntactic attributes,
better results than these word-based methods.
for similarity analysis is only productive to the
One striking commonality to these studies is the
researcher in so far as the results produced are
extent to which they are limited by the researcher’s
interpretable. Digital tools can provide new com-
ability to interpret study results. Given the previ-
prehensible interfaces that allow researchers the
ously cited indications that other features than
capacity to interpret the results based on features
the word can be used as productive features in
beyond the word.
attribution studies, it seems that the choice to use
Several studies have identified how we make such
high-frequency function words as a feature for
results more easily accessible for interpretation.
determining similarity might be based on the fact
Juola (2006) contends it is essential that tools be
that words are themselves interpretable results. For
developed, such as his JGAAP (Java Graphical
instance, John Burrows suggests that our collective
Authorship Attribution Program) prototype,
understanding of function words can provide a kind
which allows the uninitiated to try their hand at
of ground truth for measuring algorithmic accura-
authorship attribution study. He cites accuracy
cies: ‘The advantage of working with whole words’,
and usability as major concerns for such tools,
he writes, ‘rests on their accessibility and their
claiming that the only way to have better tools is
meaningfulness. They help us, in particular, to
to attract more users, who in turn are attracted by
form close and fruitful inferences about the out-
better and more usable tools (Juola et al., 2006,
come of an inquiry’ (Burrows 2002, p. 268).
p. 170). Better and more usable tools means tools
Burrows’ ideas reflect those in the data mining com-
that allow users to learn about the study of author
munity in which not being able to understand
attribution, their data sets, and the tool at the same
results is a serious limitation for the use of some
time. In other words, better and more usable (or
algorithms. The following is a description of typ-
useful) tools are tools that allow for an iterative
ical data mining procedures for developers who
interaction with the data. As we have seen, choosing
generally ‘have only a superficial understanding’
texts and features helps to fine-tune analysis, but
(Weiss et al., 2005, p. 51) of what is usually numer-
studies also show that choosing parameters such
ical data:
as weighted combinations of the best algorithms
They accept what they are given by the on the same corpus (Grieve, 2007; Sutton et al.,
domain experts and do not have a deep 2005) could advance research in author attribution.
understanding of the measurements or their Further, for instance-based similarity metrics,
relationship with each other. Results are ana- Koppel et al. (2007) argue that the ability to slice
lyzed primarily by empirical analysis. When or ‘unmask’ the results is productive. In the Koppel

584 Literary and Linguistic Computing, Vol. 28, No. 4, 2013


study, ‘unmasking’ refers to systematically dimin-
ishing the number of features for study to ‘gauge
the speed with which cross-validation accuracy de-
grades as more features are removed’ to determine
the depth of difference between texts (p. 1264). In
other words, by letting researchers slice or ‘unmask’
results in different ways, studies are strengthened.
Authorship attribution studies and stylometric ana-
lysis in general point to the fact that it is essential
when working with advanced computational simi-
larity metrics to support the user’s ability to inter-
pret the process and the results and to ask iterative
questions about which texts to choose for analysis,
which features to measure, which parameters for
analysis are the most productive, and how the re-
sults might differ when each of these aspects are
calibrated differently.
2 Changing the Software
Environment for the Advancement
of Scholarly Research Flow and
Updating ProseVis with Discovery
Similarity Metrics
This section addresses two areas of development in
this study that advance research on prosody analysis
on documents written by different authors. Using
an analysis service (Meandre) and a visualization
tool (ProseVis), this study demonstrates the afore-
mentioned iterative discovery processes for similar-
ity analysis. First, we discuss our approach, which
includes discovery techniques that bring the role
of the researcher and what counts as ‘interpretable’
results into focus. Second, we discuss our develop-
ment of Meandre and ProseVis, a coupling of data
flow and interactive visualization interface that
allows users to choose texts, a variety of prosodic
features, the size of phrases windows,1 the weighting
power, and smoothing parameters for iterative test-
ing and comparing results.
2.1 Supervised learning versus
discovery techniques
Before discussing current developments in this
study, it is useful to consider comparative results
Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Distant listening to Gertrude Stein’s ‘Melanctha’
using a supervised learning paradigm. In the super-
vised learning paradigm, the goal is to maximize
predictive accuracy. For instance, a researcher
wants to determine if Shakespeare wrote a given
text. This can be modeled as a two-class prediction
problem based on labeled examples. One class
would be all Shakespeare documents and the other
class would be documents Shakespeare did not write
from the same period and location. The perform-
ance of the system would be measured by predictive
accuracy, meaning how likely the machine learning
system can predict whether a new unseen text was
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
written by Shakespeare. Based on the set of training
examples, the machine learning system would create
a mathematical model and discover system param-
eters for achieving highest accuracy to predict the
author of unseen books. Once the system has been
trained, the resulting model can be viewed as a
‘Shakespeare’ text detector. Given a new text, it
would predict what sections are more likely to
have been written by Shakespeare. The process of
discovering the best system parameters for algo-
rithms is called bias optimization. For supervised
learning, bias optimization can be automated
based on known examples.
For this study, to extract the textual features we
need, we incorporate OpenMary, a text-to-speech
application tool for extracting aural features into
the ‘flow’ we coordinated in Meandre, a data flow
environment developed by the SEASR (Software
Environment for the Advancement of Scholarly
Research) team at the University of Illinois at
Urbana-Champaign. We developed ProseVis as a
reader interface that would allow readers to com-
pare the prosodic patterns that resulted from the
predictive modeling procedures in and across texts.
The supervised learning method we were using in
a previous study (Clement et al., 2013) quickly
became unwieldy and computationally expensive
when we sought to scale up our number of texts
of study to compare ‘Melanctha’, the longest story
in Gertrude Stein’s Three Lives (1909), with 150 dif-
ferent narrative voices from the First Person
Narratives (FPN) of the Documenting the
American South digital publishing initiative . . . In
the previous study, if a user initiated a prediction
problem, the results changed every time the user
585
T. Clement et al.
added a new document. By definition, prediction
modeling is asking a closed set question: This
phrase came from which of these particular texts?
In addition, our initial prosody research used super-
vised learning with bias optimization to determine
the best system parameters, so it was computation-
ally intensive. If the collection of documents chan-
ged, then the whole analysis would need to be run
again. For a result, it also took days to discover the
best system parameters. Besides the fact that scaling
up was computationally expensive, this method-
ology did not facilitate the kind of iterative user
interaction we were seeking. For an interactive pros-
ody analysis tool, it would have been unacceptable
to wait so long each time the collection changed.
On the other hand, within the discovery para-
digm, judging the system performance is
subjective . . . An example problem of this paradigm
would be to understand the similarities between
various authors’ writing styles. The researcher first
selects representative texts for each author and then
uses the system to measure the similarity based on
different features. Each set of different parameters
would produce different results and visualizations,
because the similarity metric could be based on
many different system control parameters, including
phrase window size, feature selection, distance
weighting power, and smoothing factors. Some ex-
periments of parameters settings may not produce
informative results, but because the researcher can
explore the effects of different similarity metrics and
learn how authors compare along these various di-
mensions, she has a better chance of discovering
ones that reveal meaningful patterns.
For this study, we implemented a similarity-
based discovery paradigm. We performed an initial
pass at comparing Three Lives with the 150 FPN
documents using sampling to identify a subset of
documents to work with in more detail. Because
our goal was to examine prosodic features, we did
not include the word or the sound in our initial
pass. We included part of speech, accent, stress,
tone, and the break index.2 In this study, each fea-
ture is represented by itself and is not combined
into a symbol. Correspondingly, each example is
represented by the phrase window size times the
number of attributes (e.g., if the user sets the
586 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
window size equal to 14 and uses accent, stress,
and tone, then 14  3 is the number of features
for each example). An example is a feature vector
describing the window in terms of the chosen fea-
tures. Distance is computed as the sum of the abso-
lute value of the differences between all features in
the feature vector.
First, we randomly chose 10,000 samples for each
of the 150 FPN documents and for each Stein text,3
with each sample comprising the five-feature set
described previously. Next, this 10,000 random
sample from each work was compared with the sam-
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
ples from each of the other documents. The similar-
ity metric we are currently using to compare phrase
windows is a form of inverse distance weighting.
Training Example Weight ¼
1=ðDistance To Testing ExampleÞ ^
Distance Weighting Power
The results of the comparison with FPN are pic-
tured in the confusion matrix in Fig. 1. This confu-
sion matrix represents a summary of the number of
samples that were like each of the documents. Each
row and column shows data about the text from
which we drew the samples. Each sample was deter-
mined to be like some other sample and those
counts increase in color darkness according to that
scale in the boxes. For all the works, the highest
prediction is for the work itself (indicated by the
dark diagonal line). Each time a new text is added
to the collection, the similarity between feature win-
dows in the new text and windows in all other texts
can be computed and added to the matrix.
The tables show lists of FPN documents and
counts based on two different perspectives of simi-
larity between FPN and Three Lives. Table 1 includes
counts of FPN samples that are higher when FPN
samples are compared with Three Lives samples.
Table 2 includes counts of Three Lives samples
when Three Lives samples are compared with FPN
samples. Because we wanted to verify the similarity
metric and to establish the style of Three Lives in
comparison with other Stein texts, we included a
few of her other works. As a result, the counts are
lower in the first table because some of the FPN works
are more similar to each other than the Stein works.
In the second table, one can see Three Lives is most
 Distant listening to Gertrude Stein’s ‘Melanctha’

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

Fig. 1 This confusion matrix shows the number of 10,000 samples that were alike in the Gertrude Stein/FPN corpus.
Each row and column shows data about the text from which we drew the samples. The highest predictions are for the
work itself (indicated by the dark diagonal line)

similar to two texts by Stein and then interwoven
between FPN corpus and the other Stein works.
The tables represent a range of documents writ-
ten by a variety of men and women from different
racial backgrounds. Specifically, in the FPN collec-
tion, there are 154 authors4: 49 are female, of whom
45 are white and 4 are former slaves; 105 are male,
of whom 77 are white and 28 are former slaves.
When we compared Three Lives with samples from
FPN, the top 10 matches (listed in Table 1) included
eight women and two slave authors. When we com-
pared samples from Three Lives with the FPN texts,
two female authors appear in the top 10 matches
and five slave authors. The system picked two of the
four slave narratives written by women for this top
list. This initial study provided an indication of
which texts to look at more closely for comparison
within the ProseVis environment.
2.2 Using the Meandre/ProseVis
discovery system
In the ProseVis webform5, the researcher is given
the opportunity to upload a selection of texts, and
control the features to use for the analysis.6 The
following are the parameters researchers can use
to control the experiment:
Comparison Range—This is comma-
separated list of indices of the documents to
be compared. For example, the user can choose
to compare just the first document with the
remaining documents in a set by using ‘1’.

Literary and Linguistic Computing, Vol. 28, No. 4, 2013 587

T. Clement et al.

Table 1 This list includes counts of FPN samples that are higher when FPN samples are compared with Three Lives
samplesa
Author #FPN like TL
Grimball, Margaret Ann Meta Morris, 1810–1881. Journal of Meta Morris Grimball: South Carolina, 73
December 1860-February 186.
Pringle, Elizabeth Waties Allston. A Woman Rice Planter. 71
Avary, Myrta Lockett. A Virginia Girl in the Civil War, 1861-1865. 69
Battle, Laura Elizabeth Lee. Forget-me-nots of the Civil War; A Romance, Containing Reminiscences and Original 69
Letters of Two Confederate Soldiers
LeConte, Joseph. The Autobiography of Joseph LeConte. 64
Dawson, Sarah Morgan. A Confederate Girl’s Diary. 63
Veney, Bethany. The Narrative of Bethany Veney: A Slave Woman. 62

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

Edmondson, Belle. Diary of Belle Edmondson, January - November, 1864 57
Zettler, B. M. (Berrien McPherson). War Stories and School-Day Incidents for the Children 51
Jacobs, Harriet A. (Harriet Ann). Incidents in the Life of a Slave Girl, Written by Herself 50
Other stein works
Stein, Three Lives 849
Stein, ‘Miss Furr and Miss Skeene’ 108
Stein, Making of Americans 71
a
There are only 5407 samples from other works that are like Three Lives (including the 849 that are from Three Lives and 108 from
‘Miss Furr’).

Table 2 This list includes counts of Three Lives samples when Three Lives samples are compared with FPN samples
Author and title #TL like FPN
Malone, Bartlett Yancey. The Diary of Bartlett Yancey Malone 598
Horton, George. The Poetical Works of George M. Horton: The Colored Bard of North Carolina: To Which is Prefixed 390
the Life of the Author, Written by Himself.
Ward, Dallas T. The Last Flag of Truce. 296
Patton, James. Biography of James Patton. 249
McLeary, A. C. Humorous Incidents of the Civil War. 220
A Georgia Negro Peon. The New Slavery in the South–An Autobiography. 215
Jones, Thomas H. The Experience of Rev. Thomas H. Jones, Who Was a Slave for Forty-Three Years. Written by a 206
Friend, as Related to Him by Brother Jones
Horton, George. The Life of George M. Horton. The Colored Bard of North Carolina. 180
Mitchel Cora. Reminiscences of the Civil War. 179
Roper, Moses. A Narrative of the Adventures and Escape of Moses Roper, from American Slavery. 157
Other stein works
Stein, Three Lives 849
Stein, Four Saints 719
Stein, ‘Matisse’ 395
Stein ‘Picasso’ 371
Stein, ‘Miss Furr and Miss Skeene’ 305
Stein, Making of Americans 178

Using ‘1, 3, 7’ means that the first, third, and
seventh documents will be compared against
each other and all of the other documents.
Using ‘all’ means that all documents will be
compared with each other.
Window Size in Sounds—This is the number
of phonemes to be considered a phrase for
analysis. Because we are working on prosodic
patterns that are affected by phrasal patterns
(Clement et al., 2013), it makes sense for this

588 Literary and Linguistic Computing, Vol. 28, No. 4, 2013


value to represent the average number of
sounds in a phrase. If texts use shorter phrases,
then a smaller window serves as a better repre-
sentation of the average phrase size for a
given text. If texts have longer phrases, then a
larger window might yield more productive
results.
Sound Features to Use—This refers to the at-
tributes of the sounds, which are determined
from the features extracted by OpenMary, a
pre-processing module in Meandre. As
described in Clement et al. (2013), this
module uses the OpenMary text-to-speech
software. Attributes are explained in detail in
the OpenMary Documentation, specifically at
http://mary.dfki.de/documentation/module-
architecture/. The features can be used one at a
time or in combination. The sound features
that we are using include (1) part of speech;
(2) accent, which indicates the pitch of the
sound; (3) stress, which indicates the presence
of a primary or secondary lexical stress;
(4) break index, which indicates when sounds
precede phrase breaks, sentence breaks, and
paragraph final breaks; and (5) tone, which
indicates the location of prosodic boundaries
and pitch accents by assigning sentence type
(e.g., declarative, interrogative-W, interroga-
tive-Yes-No, and exclamatory).
Weighting Power—This number radically
controls the behavior of the instance-based
learner. Valid values are in the range 0 to
100. When weighting power is set to the high-
est value, it heavily weights close matches when
computing similarity. When set to the lowest
value, it equally weights all matches. Higher
weighting power values caused our instance-
based learning system to use a nearest-
neighbor strategy. With lower values, more
weight is given to distant examples, which ef-
fectively increases the neighborhood size.
When set to zero, all training examples are
equally weighted, resulting in a constant pre-
diction reflecting the baseline class
probabilities.7
The last parameter the researcher may control is
the smoothing factor. Smoothing is a technique
Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Distant listening to Gertrude Stein’s ‘Melanctha’
used in image processing and statistics to ‘blur’
out more detailed features and emphasize the
larger scale features. In ProseVis, smoothing is
used to find longer patterns by averaging the simi-
larity values over a neighborhood. Using the data
produced through Meandre to compute document
similarity based on prosody features, ProseVis allows
researchers to explore these results mapped back to
the original text with colors. By default, Meandre
returns a collection of raw similarity values on a
per-syllable-per-document basis that is often too
small to display without some form of normaliza-
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
tion. The ProseVis tool ensures that all similarity
values are scaled according to their relative weights
in the interval [0.1] by determining the global max-
imum and minimum similarity values and scaling
each similarity value ‘v’ by the function (v-min)/
(max-min). Within ProseVis we determine the rela-
tive weight of each document per syllable and
choose the highest value for coloring. The value
only takes into account all comparison values
among documents that are selected. Smoothing is
necessary to take into account the context of the
syllable with respect to its neighbors. The smoothing
function averages the similarity values of the N
neighbors to the left and to the right of the syllable
with a window size of 2  N þ 1, thus incorporating
the notion of context during this coloring
step. While in a supervised learning system, the
algorithm can adjust the smoothing factor to
achieve maximum predictive accuracy. In a dis-
covery system, the researcher can adjust the
smoothing factor to maximally reveal the features
of interest.
The following figures show examples of results
based on changing all of the aforementioned param-
eters. Figure 2 shows the ProseVis interface. In this
image, each sound in ‘Melanctha’ is colored accord-
ing to the document to which the system has given
that sound the highest similarity value. What is sig-
nificant in the findings is that particular FPN docu-
ments seem to map in surprisingly regular patterns
to certain parts of ‘Melanctha’. Specifically, the
green blocks shown in Fig. 4 indicate sections of
the text that the system has determined sound
most similar to The Diary of Bartlett Yancey
Malone (1919), written by a Confederate farmer
589
T. Clement et al.
Fig. 2 ‘Melanctha’ colored in ProseVis according to similarity with FPN documents listed in Fig. 3. Color versions of all
figures are available in the online version of the paper
turned soldier and sergeant from North Carolina.
The blue blocks indicate parts of ‘Melanctha’ that
the system has determined sound most similar to
The Poetical Works of George M. Horton: The
Colored Bard of North Carolina: To Which is
Prefixed the Life of the Author, written by himself
(1845). The colors range in intensity based on the
value of the similarity value. Figure 3 shows the
tool panel where a user can see which colors corres-
pond to the texts in this similarity study and the
check boxes that allow a user to deselect a text
and remove it from the comparison. For in-
stance, if the Horton document (labeled here as
‘hortonpoem’) were deselected, the blocks that are
blue would change to reflect the color of the text
with the next highest similarity metric. Examples of
this ‘unmasking’ are included in the third section
of this article.
First, we tested various parameters such as
weighting. Figure 4 below shows three panels,
each showing ‘Melanctha’ from Three Lives. These
results are based on using a phrase window of 14
590 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
sounds with similarity analysis using the features
discussed earlier—part of speech, accent, tone,
stress, and break index. Each panel shows a differ-
ent weighting power, from left to right: these are
16, 32, and 64. Although the blocks of colors (pri-
marily blue and green) are the same in all three
panels, the left panel shows larger blocks of color
than the panel on the far right, where the colors
are more varied. This might indicate that to exam-
ine the texts in this sample for longer textual pat-
terns (i.e., multi-phrasal blocks, sentences, or
paragraphs) that make sense to readers, the lower
weighting power will yield more productive
visualizations.
Figure 5 is also a comparison of three versions of
results on Three Lives. In this view, the researcher
has chosen to differentiate which features to choose.
Each panel includes results produced using the
14-sound window and 16 for a weighting power.
The difference here is that the first panel includes
all the features used previously, the second includes
all but break index, and the third contains all but
 Distant listening to Gertrude Stein’s ‘Melanctha’

part of speech. These results show that including

both parts of speech and break index is important
to produce more productive results.
Figure 6 shows two panels that contain the
same excerpt from ‘Melanctha’ based on results
from similarity analysis using a 14-sound phrase
window and a 16 weighting power. The first panel
has a smoothing value of ‘1’, whereas the panel on
the right has a smoothing value of ‘15’. In this case,
the smoothing value of ‘15’ emphasizes the differ-
ently colored blocks and helps to facilitate the user’s
interpretation of the results.

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

Fig. 3 From the ProseVis control panel, the list of FPN
authors being compared with Three Lives and their asso-
ciated colors
These examples show the different kinds of
results that researchers can produce within the
iterative discovery-based paradigm that ProseVis
project facilitates. In the later discussion, using the
14-sound phrase window with a weighting power
of 16 and a smoothing factor of 15, we demonstrate
how ‘unmasking’ the data in the ProseVis interface
is productive for analyzing identity construction in
Gertrude Stein’s short story ‘Melanctha’.

Fig. 4 Three ProseVis panels, each with an excerpt from ‘Melanctha’ showing based on a 14-sound window and
different weighting powers from left to right: 16, 32, and 64

Literary and Linguistic Computing, Vol. 28, No. 4, 2013 591

T. Clement et al.
Fig. 5 Excerpt from ‘Melanctha’ colored in three ProseVis panels based on a 14-sound phrase window and 16 for a
weighting power; the first panel includes all the features used previously, the second includes all but break index, and
the third contains all but part of speech
3 Distant Listening to Gertrude
Stein’s ‘Melanctha’
Gertrude Stein scholars have written extensively
about the influence that African American speech
patterns may have had on Stein’s style of writing
during the period in which she wrote ‘Melanctha’,
the longest ‘life’ or short story in her book Three
Lives. Some scholars find her treatment of race in
‘Melanctha’ to be ‘pernicious’ (Fullbrook, 1990,
p. 69) or ‘vicious’ (Saldı́var-Hull, 1989, p. 190).
Milton Cohen (1984) creates a chart that organizes
the characters ‘into a racial hierarchy that is [. . .]
ominously schematic’ (p. 119). Other scholars such
as Richard Bridgman (1970) and writer Claude
McKay (quoted in Brinnon, 1959, p. 121) find the
characteristics of Melanctha’s friends and family to
be stereotypes and caricatures that have little to do
with the story. Still others, such as Carla Peterson
(1996) and Lorna Smedman (1995), complicate
592 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
these readings by suggesting that the relationships
between form and content, style and philosophy,
and aesthetics and politics in ‘Melanctha’ create a
more variegated look at identity construction
(including race and gender) in this text.
Peterson and Smedman’s readings of ‘Melanctha’
are of particular interest in this study because they
indicate that the prosodic elements of Stein’s
‘Melanctha’ point to ‘shared’ racialized and gendered
identities that cannot be easily classified. Peterson,
who contends that Stein’s ‘inspiration derive[d]
quite specifically from Baltimore and, in the case of
[the story of] ‘‘Melanctha,’’ from African American
Baltimore’, (1996, p. 141) identifies repetitive phrasal
patterns that evoke syncopated rhythms much like
the ragtime music that was popular in Baltimore at
the time of its writing. Of significance to this study
are Peterson’s claims that ‘Melanctha’ captures the
blurred racial identities, the ‘complex racial border-
land’ of ‘early twentieth century America where

Fig. 6 Excerpt from ‘Melanctha’ colored in three ProseVis panels based on a 14-sound window and 16 for a weighting
power
blood lines are often blurred and cultural traditions
merged’ (p. 144). Specifically, Peterson argues, Stein
captures this perspective by appropriating African
American musical traditions—coon songs, early
folkblues, and ragtime music—with her prosody
that historically have been ‘inextricably bound’ to a
variety of American ethnicities and cultural back-
grounds. In addition, Peterson and Smedman point
to Stein’s ‘double identity as a Jew and a lesbian’
(Peterson, 1996, p. 155) as a thematic element in
the text that is also inscribed in its racial discourse,
specifically in its work to investigate racialized signi-
fiers (Smedman, 1995, p. 570). ‘Since Stein’s linguis-
tic tampering involved an erotics and experience
outside of the normative heterosexual boundaries’,
Smedman writes, ‘it is not surprising that she makes
the link between ‘‘improper’’ racialized language and
‘‘taboo’’ sexuality so often in these texts’ (p. 571). In
other words, these critics are arguing that Stein uses
stylistic features—specifically prosodic elements—to
Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Distant listening to Gertrude Stein’s ‘Melanctha’
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
work with thematic and narrative elements to create
an interplay between sounds and syntax that gestures
toward a story of shared cultures.
Continuing with the vein of inquiry suggested by
Peterson and Smedman, we compare Gertrude
Stein’s ‘Melanctha’ with 150 FPN of the American
South collection to interrogate how the system meas-
ures the extent to which Stein’s ‘Melanctha’ sounds
like or contains prosodic elements similar to those
found in these narratives. Self-described, the FPN ‘is
a collection of diaries, autobiographies, memoirs,
travel accounts, and ex-slave narratives written by
Southerners. The majority of materials in this collec-
tion are written by those Southerners whose voices
were less prominent in their time, including African
Americans, women, enlisted men, laborers, and
Native Americans’.8 Even though ‘Melanctha’ is writ-
ten from the third-person perspective, it is written in
the free indirect style. In the free indirect style, a
character’s way of speaking, either out loud or in
593
T. Clement et al.
his or her thoughts, dictates the style of narration,
making the narrative much like a first-person narra-
tive. Using ProseVis to distant-listen to ‘Melanctha’
by comparing its prosodic elements with those in the
FPN documents allows for new readings of the text’s
portrayal of identity construction as it corresponds
to the sound of the text.
3.1 Discussion: Unmasking the sound of
identity construction in ‘Melanctha’
This study’s driving question is not to ask the ques-
tion ‘Does ‘‘Melanctha’’ sound like narratives writ-
ten by African Americans and women written in
approximately the same time period?’ because such
a question is not only rendered mute by questions
concerning the authenticity of these narratives (who
wrote them) but also because first-person narratives
or autobiographies are politicized and embodied
documents, and as such, make slippery signifiers.
As Sidonie Smith writes, an autobiography ‘histori-
cizes identity implicitly, if not explicitly, insists on
the temporalities and spatialities of identity and, in
doing so, brings the everyday practices of identity
directly into the floodlights of conscious display’
(1993, p. 160). In other words, the FPN collection,
filled with romanticized stories of soldiers at war,
widows and wives and daughters at home, and
slaves who were abused and demoralized and
escaped, is not representative of writing that signifies
race or a gender as much as it is a collection that
shows a set of writers expressing or practicing iden-
tity (which was often racialized and gendered) at a
certain time and in a certain geography, specific-
ally in the South at the end of the 19th century.
Similarly, this study shows that the system, which
is not aware of gender and race, can be sensitive to
the ‘masks’ that gendered and racialized language
can assume.
The next set of images shows the results of our
study visualized in ProseVis as well as the similarity
patterns that a researcher can ‘unmask’ on key para-
graphs in ‘Melanctha’. In the paragraphs shown in
the following figures, Melanctha, who is described as
‘a graceful, pale yellow, intelligent, attractive
negress . . . half made with real white blood’ (Stein,
2004, p. 58), and her boyfriend, Dr. Jeff Campbell,
who is described as ‘an intelligent good mullato’
594 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
(p. 86), are facing the denouement of their relation-
ship, the building of which has formed the central
narrative of the story. After this point, their rela-
tionship begins to unravel. The break up has been
foreshadowed: from the beginning of the story, Jeff
Campbell ‘did not like Melanctha’s ways,’ and ‘he
did not think that she would ever come to any good’
(Stein, 2004, p. 77). Melanctha’s ‘way’ through the
text is to ‘wander’ both sexually and intellectually,
what the narrator calls ‘wandering after wisdom’. At
the point of the text shown in Fig. 7, Jeff has heard
more rumors about Melanctha’s past from
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
Melanctha’s former lover Jane Harden and yet he
and Melanctha have had ‘much joy between them,
more than they ever yet had had with their new
feeling. All the day they had lost themselves in
warm wandering. Now they were lying there and
resting’ (Stein, 2004, p. 105). After this encounter,
Jeff suddenly ‘threw Melanctha from him’ (p. 106).
Feeling guilty, Jeff explains to Melanctha that when
he met her, he only knew ‘two kinds of way of
loving, one way the way it is good to be in families
and the other kind of way, like animals are all the
time just with each other, and how I didn’t ever like
that last kind of way much for any of the colored
people’ (p. 107). Melanctha, he explains, has shown
him a third way of living that is ‘what really loving is
like’ (p. 107), but Melanctha, he determines here
after a day of intense ‘wandering’, is ‘a bad one’.
Melanctha’s feelings are hurt and they talk and con-
tinue their relationship, but this cycle of sex, retri-
bution, and unraveling trust continues until they
part for good at the end of the story.
What is significant about this narrative pattern
that seems to correspond with Jeff’s ‘two kinds of
way’ is the extent to which the prosodic patterns we
can see in ProseVis in Fig. 2 and Fig. 4 mirror the
same kind of vacillation between two kinds of nar-
ration in the story, one that comprises short,
clipped, and simple sentences on the one hand,
which usually map in similarity to the Malone docu-
ment (in green), and long, multi-phrasal, and com-
plex sentences on the other, which usually map to
the Horton document (in blue). Often but not
always, the green clipped text maps to moments in
the story that correspond to Jeff’s actions when he is
not consciously thinking about Melanctha or when
he is feeling negatively towards her and her

Fig. 7 ‘Melanctha’ excerpt colored in ProseVis according to similarity with FPN documents listed in Fig. 3
wandering ways. The following is an example of a
green-colored section:
She began to tell everything she ever knew
about you. She didn’t know how well now
I know you. I didn’t tell her not to go on
talking. I listened while she told me everything
about you (Stein, 2004, p. 102).
The blue maps to the more loose and multi-phrasal
text that corresponds to either a description of
Melanctha’s actions and thoughts or Jeff’s when he
is feeling positive or affected by Melanctha. The fol-
lowing is an example of a blue-colored section:
I see that now, sometimes, the way you cer-
tainly been teaching me, Melanctha, really,
and then I love you those times, Melanctha,
like a real religion, and then it comes over me
all sudden, I don’t know anything real about
you Melanctha, dear one, and then it comes
over me sudden, perhaps I certainly am wrong
now, thinking all this way so lovely, and
not thinking now any more the old way
I always before was always thinking, about
Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Distant listening to Gertrude Stein’s ‘Melanctha’
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
what was the right way for me, to live
regular . . . (Stein, 2004, p. 108)
Further, with a wider view of the entire story as visua-
lized in Fig. 2, the researcher can see that the begin-
ning of the story, which corresponds to the narrative
of Melanctha’s upbringing and the maturing and so-
lidifying of her ‘wandering’ ways, is predominately
like Horton’s document, whereas the end of the text,
which corresponds with Melanctha’s decline into
despondency and ultimately sickness, the ceasing of
her wandering, shows more similarity with Malone.
At first glance at these patterns, it would seem that we
could make a simple assertion that the system found
the slave narrative (Horton’s document) to sound
more like Melanctha’s wandering narrative, whereas
it found the narrative corresponding to Jeff’s way of
thinking to sound more like that told by the soldier
(Malone’s document).
As discussed earlier, however, interacting with
the data is an important advancement in similarity
testing. By ‘unmasking’ (Koppel et al., 2007) or de-
selecting texts in ProseVis, the researcher can quickly
see how these similarity patterns are more complex.
595
T. Clement et al.
For example, if the researcher starts with selecting all
of the texts as shown in Fig. 4, the green and blue
blocks are evident.
In Fig. 8, the researcher has deselected the blue
Horton text to reveal a purple pattern that indicates
similarity with The Narrative of Bethany Veney: A
Slave Woman (1889). This is because for the same
section of the text, the next highest value for these
phrases corresponds to the Veney document. The
green block remains mostly unchanged.
In Fig. 9, both Horton and Veney are deselected
and a pale purple is revealed, indicating that the
document with the next highest number of likeness
corresponds with Sarah Morgan Dawson’s A
Confederate Girl’s Diary (1913).
Unchanged by these unmaskings of the blue
blocks, the green blocks indicate a different relation-
ship of similarities. In Fig. 10, the researcher has
deselected the green Malone document and finds a
pale red pattern that indicates that the next highest
numbers of similarity for this part of the text cor-
respond to A.C. McLeary’s Humorous Incidents of
the Civil War (1902).
In Fig. 11, the researcher has deselected McLeary
to unmask a pale green that indicates a similarity to
Fig. 8 The Horton (blue) document comparison has been deselected to reveal the Veney (purple) document similarity
596 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Dallas T. Ward’s The Last Flag of Truce, the story of
a railroad conductor or merchant (the history is
unclear) who was asked to make the Confederates’
truce flag of surrender.
To summarize, by unmasking the blue Horton
document (written by a male slave), the researcher
reveals the purple Veney document (written by a
female slave) and then the pale purple Dawson
document (written by a female Confederate). By
unmasking the green Malone document (written
by a male Confederate), the researcher reveals the
pink McLeary document (written by a male
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
Confederate) and the pale green Ward document
(written by a male Confederate).
These initial discoveries into the similarities be-
tween ‘Melanctha’ and the FPN narratives invite the
researcher to consider two new research questions
about identity construction in Gertrude Stein’s
‘Melanctha’. Smith writes that autobiographical nar-
ratives ‘carry with them through these negotiations
the specificities of their material circumstances, their
degrees of self-consciousness about cultural deter-
mination, the temporalities of their bodies’ (Smith,
1993, p. 22). A first research question that is pro-
voked by this discovery experience might consider
 Distant listening to Gertrude Stein’s ‘Melanctha’

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

Fig. 9 Both the Horton and Veney document comparisons are deselected and the Dawson similarity (pale purple) is
revealed

Fig. 10 The Malone document comparisons have been deselected and the McCleary similarity (pink) is revealed

Literary and Linguistic Computing, Vol. 28, No. 4, 2013 597

T. Clement et al.

Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015

Fig. 11 Both the Malone and McLeary document comparisons have been deselected and the Ward similarity (pale
green) is revealed

the ‘temporalities of identity’ on ‘conscious display’
(Smith, 1993, p. 160) in the FPN and the ‘Melanctha’
documents. For instance, the blue group authors all
taught themselves to write. Only Dawson had 10
months of formal schooling. The blue group docu-
ments are also self-proclaimed ‘literary’ documents.
Horton, who was the first African American to pub-
lish a book in the American South, wrote poetry. His
document is primarily a book of poetry with a long
personal narrative as an introduction. Veney’s nar-
rative, a tract that is meant to illustrate Veney’s
Christian character to a Reconstruction-era reader-
ship still reeling from the war, is introduced by Rev.
Bishop Mallalieu, and includes ‘Commendatory
Notices from Rev. V. A. Cooper, Superintendent of
Home for Little Wanderers, Boston, Mass., and Rev.
Erastus Spaulding, Millbury, Mass’. The Rev. Bishop
Mallalieu writes:
It is greatly to be regretted that the language
and personal characteristics of Bethany cannot
be transcribed. The little particulars that give
coloring and point, tone and expression, are
largely lost. Only the outline can be given. As
it is, possessing only the merit of a ‘‘plain,
unvarnished tale,’’ it asks for generous consid-
eration and extended sale.
Taken at face value, this comment would seem to
indicate that Veney’s document was written in a
plain style, but the intent of the text is to convince
the audience that ‘the biographies of saintly, endur-
ing spirits like that of Betty Veney will be read, and
will serve to inspire the discouraged and down-trod-
den to put their trust in the almighty arm of
Jehovah’, and it is clear that the bishop’s introduc-
tion is meant to quell concerns that such a tale, writ-
ten in a style to inspire empathy and religious
zeal, might be untrue. Finally, the last of the blue
group documents by Dawson is introduced by a
long introduction from her son, who describes the
narrative’s ‘flowing sentences’, its ‘certain uses of
words to which the twentieth century purist will
take exception’, and its likeness to Victorian litera-
ture as a ‘remarkable feat of style’ (p. xii). The au-
thors’ backgrounds and the assumed audiences for
the green group of documents are remarkably differ-
ent. Two of the writers (Malone and Ward) are

598 Literary and Linguistic Computing, Vol. 28, No. 4, 2013


soldiers. Both Malone and McLeary’s diaries are re-
ports of daily happenings. Malone’s is described as
‘Reported in a simple and matter-of-fact manner,
include notations on his diet, his regiment’s marches,
and biblical texts referred to in the sermons he hears’.
Ward’s tale is also matter-of-fact and dedicated ‘To
the Soldiers’. It is introduced with letters from a
businessman and a judge who attest to its veracity.
The similarities that tie the ‘blue group’ documents
(Horton, Veney, and Dawson) together and the
‘green group’ documents (Malone, McLeary,
Ward) together to the same spots in ‘Melanctha’
are also based on stylistic differences that are the
direct result of the authors’ educational backgrounds
and their intended audiences, which are evidenced
by the introductions and introductory letters that
accompany most of these writings.
A second question the researcher is provoked to
consider concerns the presence of three FPN docu-
ments that appear regularly in intermittent patterns
across ‘Melanctha’. These are Harriet Jacobs’ text
Incidents in the Life of a Slave Girl, written by herself
(1861) in orange; Margaret Ann Morris Grimball’s
Journal of Meta Morris Grimball: South Carolina,
December 1860-February 1866 in red; and Elizabeth
Waties Allston Pringle’s A Woman Rice Planter
(1914) in gray. All three have a comparable similar-
ity metric based on 10,000 samples (Table 1) but
never appear as solid blocks in the text as the
green group and blue group patterns do. Instead,
they have a sporadic but constant appearance in
the text across the other blocks of color. It is these
constant underlying patterns in Stein’s texts that
often tell much of the story. The presence of these
patterns would seem to suggest that further investi-
gation into the similarities between these documents
and the essentially mixed nature of identity and
textual construction in Stein’s Three Lives would
be productive.
4 Conclusion
A third research question this study of ‘Melanctha’
provokes for consideration is the extent to which
this work furthers similarity analysis in author attri-
bution studies. A form of attribution that continues
Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Distant listening to Gertrude Stein’s ‘Melanctha’
to elude researchers who work in attribution studies
corresponds to the mixed borderline of racialized
and gendered identity construction to which
Peterson and Smedman refer. Like voices and iden-
tities constructed with mixed histories and mixed
influences, texts are often the result of collaborative
authoring. The lack of studies that consider this
aspect of texts in authorship attribution has been
described as ‘a pitfall’ common to attribution stu-
dies (Eder, 2012). Eder contends that the future of
such study rests in using ‘stylometric techniques to
trace stylistic imitations or unconscious inspirations
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
between different authors’ (Eder, 2012) and Collins
et al. (2004) pushes the boundaries of these limits by
attempting to characterize[e] the Federalist Papers
‘according to the representational language choices
of the authors, similar to a way we believe close
human readers come to know a text and distinguish
its rhetorical purpose’ (Collins et al., 2004, p. 15).
Specifically, Collins uses the frequencies of ‘repre-
sentational language strings’ he has identified that
indicate ‘subtle rhetorical impressions’ (Collins
et al., 2004, p. 15). He uses these strings to differ-
entiate and attribute parts of the Papers to Hamilton
and Madison. Collins writes that these quantitative
measurements help to facilitate nonquantifiable as-
pects of collaborative writing and influence.
Our study reflects the development of a system
for detecting similarity that also allows the re-
searcher to assert her understanding of the text as
she reads or listens to it, from a distance and up
close, in an iterative fashion. Stein writes of her syn-
esthetic relationship with texts this way: ‘I feel with
my eyes and it does not make any difference to me
what language I hear, I don’t hear a language, I hear
tones of voices and rhythms, but with my eyes I see
words and sentences’ (Stein, 1990, p. 70); and, in
another piece, she wonders, ‘Did one see sound, and
what was the relation between color and sound, did
it make itself by description by a word that meant it
or did it make itself by a word in itself’ (Stein, 1988,
p. 191). Likewise, the ProseVis project creates a space
for distant listening by establishing relationships be-
tween sound and color, color and text, and text and
sound. It at once quantifies and unifies modes of
signification, allowing for a new perspective on
how we make meaning. Charles Bernstein writes
599
T. Clement et al.
that close listening remind us that ‘individual im-
pulses need substantiality before unifying them can
generate much dynamism’ and ‘the near language-
like qualities of the musics of writing . . . gives them
an outwardly blinking and scanning and surfing
involvement with a body politic or political economy
of sense’ (Bernstein, 1998, p. 83). Likewise, tools that
provide for readerly interactions such as the kind of
distant (and close) listening (and reading) we have
outlined here can advance the sensitivity of systems
that use algorithms such as similarity metrics, but
more importantly, we advance researchers’ under-
standings of the how and the what and the whole
those blinking metrics represent.
Funding
This work was supported by the Andrew W. Mellon
Foundation [grant number 31000682].
References
Baayen, R., van Halteren, H., and Tweedie, F. (1996).
Outside the cave of shadows: Using syntactic annota-
tion to enhance authorship attribution. Literary and
Linguistic Computing, 11: 121–31.
Bernstein, C. (1998). Close Listening: Poetry and the
Performed Word. New York: Oxford University Press.
Bridgman, R. (1970). Gertrude Stein in Pieces. New York:
Oxford University Press.
Burrows, J. F. (1987). Computation into Criticism: A
Study of Jane Austen’s Novels and an Experiment in
Method. Oxford: Clarendon Press.
Burrows, J. (2002). ‘Delta’: A measure of stylistic differ-
ence and a guide to likely authorship. Literary and
Linguistic Computing, 17: 267–87.
Brinnon, J. M. (1959). The Third Rose: Gertrude Stein and
Her World. New York: Atlantic-Little Brown.
Cohen, M. A. (1984). ‘‘Black Brutes and Mulatto Saints:
The Racial Hierarchy of Stein’s ‘Melanctha’ ’’. Black
American Literature Forum, 18(3): 119–21. Autumn.
Collins, J., Kuafer, D., Vlachos, P., Butler, B., and
Ishizaki, S. (2004). Detecting collaborations in text
comparing the authors’ rhetorical language choices in
the federalist papers. Computers and the Humanities,
38(1): 15–36.
600 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Clement, T., Tcheng, D., Auvil, L., Capitanu, B., and
Monroe, M. (2013). Sounding for Meaning: Using
Theories of Knowledge Representation to Analyze Aural
Patterns in Texts. Digital Humanities Quarterly 7.1.
Dawson, S. M. (1913). A Confederate Girl’s Diary.
Cambridge, MA: The Riverside Press. Documenting
the American South. http://docsouth.unc.edu/fpn/
dawson/menu.html (accessed 11 November 2012).
Diederich, J., Kindermann, J., Leopold, E., and
Paass, G. (2000). Authorship attribution with support
vector machines. Applied Intelligence, 19(1–2):
109–23.
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
Eder, M. (2012). Mind your corpus: systematic errors in
authorship attribution. In Digital Humanities Book of
Abstracts. Hamburg: Annual Digital Humanities
Conference. http://www.dh2012.uni-hamburg.de/
conference/programme/abstracts/mind-your-corpus-
systematic-errors-in-authorship-attribution/ (accessed
November 11, 2012).
Forsyth, R. and Holmes, D. (1996). Feature-finding for
text classification. Literary and Linguistic Computing,
11(4): 163–74.
Fullbrook, K. (1990). Free Women: Ethics and Aesthetics
in Twentieth-Century Women’s Fiction, 1st edn.
Philadelphia, PA: Temple University Press.
Gamon, M. (2004). Linguistic correlates of style: Authorship
classification with deep linguistic analysis features.
Proceedings of the 20th International Conference on
Computational Linguistics. Morristown, NJ: Association
for Computational Linguistics, pp. 611–17.
Grieve, J. (2007). Quantitative authorship attribution: an
evaluation of techniques. Literary Linguistic Computing,
22(3): 251–70.
Hoover, D. L. (2003a). Another perspective on vocabulary
richness. Computers and Humanities, 37(2): 151–78.
Hoover, D. L. (2003b). Multivariate analysis and the
study of style variation. Literary Linguistic Computing,
18(4): 341–60.
Horton, G. M. (1845). The Poetical Works of George M.
Horton: The Colored Bard of North Carolina: To
Which is Prefixed the Life of the Author, Written by
Himself. Hillsborough, NC: Documenting the
American South. Printed by D. Heartt. http://doc
south.unc.edu/fpn/hortonpoem/menu.html (accessed
11 November 2012).
Jockers, M. L. and Daniela, M. W. (2010). A comparative
study of machine learning methods for authorship
attribution. Literary and Linguistic Computing, 25.2:
215–24.

Juola, P. (2004). Ad-hoc authorship attribution competi-
tion. Proceedings of the Joint Conference of the
Association for Computers and the Humanities and the
Association for Literary and Linguistic Computing.
Goteborg, Sweden, pp. 175–6.
Juola, P., Sofko, J., and Brennan, P. (2006). A prototype
for authorship attribution studies. Literary Linguistic
Computing, 21(2): 169–78.
Koppel, M., Schler, J., and Bonchek-Dokow, E. (2007).
Measuring differentiability: unmasking pseudonymous
authors. Journal of Machine Learning Research, 8:
1261–76.
Malone, B. Y. (1919). The Diary of Bartlett Yancey
Malone. Chapel Hill: University of North Carolina.
Documenting the American South. http://docsouth.
unc.edu/fpn/malone/menu.html (accessed 11
November 2012).
Margaret Ann Meta Morris Grimball. (1818–1881).
Journal of Meta Morris Grimball: South Carolina,
Deceber 1860-February 1866. TS. UNC-Chapel Hill,
Southern Historical Collection.
Martindale, C. and Mckenzie, D. (1995). On the utility of
content analysis in author attribution: The Federalist.
Computers and the Humanities, 29(4): 259–70.
McLeary, A. C. (1902). Humorous Incidents of the Civil
War [n.p.]. Documenting the American South. http://
docsouth.unc.edu/fpn/mcleary/menu.html (accessed
11 November 2012).
Moretti, F. (2000). Conjectures on World Literature. New
Left Review, (1):54–68.
Mosteller, F. and Wallace, D. (1964). Inference and
Disputed Authorship: The Case of the Federalist Papers.
Reading, MA: Addison-Wesley.
Peterson, C. L. (1996). The Remaking of Americans:
Gertrude Stein’s ‘Melanctha’ and African-American
Musical Traditions. In Wonham, H. B. (ed.),
Criticism and the Color Line: Desegregating American
Literary Studies. New Brunswick, NJ: Rutgers UP,
pp. 140–57.
Pringle, E. W. A. (1914). A Woman Rice Planter. C.
1913. New York: The Macmillan Company.
Documenting the American South. http://docsouth.
unc.edu/fpn/pringle/menu.html (accessed 11
November 2012).
Sanderson, C. and Guenter, S. (2006). Short text author-
ship attribution via sequence kernels, Markov chains
and author unmasking: An investigation. In
Proceedings of the International Conference on
Empirical Methods in Natural Language Engineering.
Literary and Linguistic Computing, Vol. 28, No. 4, 2013
Distant listening to Gertrude Stein’s ‘Melanctha’
Morristown, NJ: Association for Computational
Linguistics, pp. 482–491.
Saldı́var-Hull, S. (1989). ‘Wrestling Your Ally: Stein,
Racism, and Feminist Critical Practice’. In Lynn, M. B.
and Ingram, A. (eds), Women’s Writing in Exile. Chapel
Hill, NC: University of North Carolina, pp. 181–98.
Smedman, L. (1995). ‘‘Cousin to Cooning’’: Relation,
Difference, and Racialized Language in Stein’s
Nonrepresentational Texts. MFS Modern Fiction
Studies, 42: 569–588.
Smith, S. (1993). Subjectivity, Identity, and the Body:
Women’s Autobiographical Practices in the Twentieth
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
Century. Bloomington: Indiana University Press.
Stamatatos, E., Fakotakis, N., and Kokkinakis, G.
(2000). Automatic text categorization in terms of
genre and author. Computational Linguistics, 26(4):
471–495.
Stamatatos, E., Fakotakis, N., and Kokkinakis, G.
(2001). Computer-based authorship attribution with-
out lexical measures. Computers and the Humanities,
35(2): 193–214.
Stamatatos, E. (2009). A survey of modern authorship
attribution methods. Journal of the American
Society for Information Science and Technology, 60(3):
538–56.
Stein, G. (1990). The Autobiography of Alice B. Toklas.
New York: Vintage Books.
Stein, G. (1988). ‘‘Portraits and Repetition’’. Lectures in
America. London: Virago, pp. 165–206.
Stein, G. (2004). Three Lives. Whitefish, Montana:
Kessinger Publishing.
Sutton, C., Sindelar, M., and McCallum, A. (2005).
Feature bagging: Preventing weight undertraining in
structure discriminative learning. In Structured
Discriminative Learning. CIIR Technical Report.
Amherst, MA: University of Massachusetts.
Uzuner, O. and Katz, B. (2005). A comparative study of
language models for book and author recognition.
Lecture Notes in Computer Science. Berlin: Springer.
Veney, B. (1889). The Narrative of Bethany Veney: A
Slave Woman. Boston: Press of Geo. H. Ellis.
Documenting the American South. http://docsouth.
unc.edu/fpn/veney/menu.html (accessed 11
November 2012).
Ward, D. T. (2012). The Last Flag of Truce. Franklinton,
NC: D.T. Ward. Documenting the American South.
http://docsouth.unc.edu/fpn/ward/menu.html (ac-
cessed 11 November 2012).
601
T. Clement et al.
Weiss, S. M., Indurkhya, N., Zhang, T., and
Damerau, F. (2005). Text Mining: Predictive Methods
for Analyzing Unstructured Information. New York:
Springer.
Yu, B. (2008). An evaluation of text classification methods
for literary study. Literary and Linguistic Computing, 23:
327–43.
Zhang, D. and Lee, W. S. (2006). Extracting key-sub-
string-group features for text classification. In
Proceedings of the 12th Annual SIGKDD International
Conference on Knowledge Discovery and Data Mining.
New York: ACM Press, pp. 474–83.
Zhao, Y. and Zobel, J. (2005). Effective and scalable
authorship attribution using function words. Lecture
Notes in Computer Science. Berlin: Springer.
Notes
1 In this study, we use the term ‘phrase window’ to
mean a window of sounds that have been produced
in the pre-processing stage. The size of the window,
which in our system defaults to ‘8’, can be set by
the user.
2 The break index, which marks the boundaries of
syntactic units such as an intermediate phrase break,
an intra-sentential phrase break, a sentence-final
602 Literary and Linguistic Computing, Vol. 28, No. 4, 2013
boundary, and a paragraph-final boundary, is particu-
larly important because phrasal boundaries determine
the rise and fall or emphases of particular words based
on their context within the phrase.
3 The texts by Gertrude Stein include Four Saints in
Three Acts, ‘Matisse’, The Making of Americans, ‘Miss
Furr and Miss Skeene’, ‘Picasso’, Three Lives, and
Tender Buttons. All of these texts are freely available
online from Project Gutenberg. The Making of
Americans edition was published by Dalkey Archive
Press (1995).
4 Some of the FPN documents have multiple authors.
We do not include illustrators in this count.
Downloaded from http://llc.oxfordjournals.org/ at University of Pittsburgh on January 14, 2015
5 The webform can be found at http://tclement.ischool.
utexas.edu/ProseVis/data/.
6 We have created a Meandre service for this
backend analysis. Once the analysis is complete, the
researcher receives an email with urls to download
the results.
7 An important characteristic is determining the relation-
ship between the number of texts a user is exploring
and the window size in sounds. We have determined
that as the number of examples increases, so too should
the weighting power. Viewed from a k-nearest-
neighbor perspective, this amounts to keeping the
neighborhood size constant.
8 http://docsouth.unc.edu/fpn/index.html