Computer-Assisted Literary Translation (Routledge Advances in Translation and Interpreting Studies Series)

Computer-Assisted Literary Translation
This collection surveys the state of the art of computer-assisted literary

translation (CALT), making the case for its potential to enhance literary
translation research and practice.
The volume brings together early career and established scholars from
around the world in countering prevailing notions around the challenges
of effectively implementing contemporary CALT applications in literary
translation practice which has traditionally followed the model of a single
translator focused on a single work. The book begins by addressing key
questions on the definition of literary translation, examining its sociological
dimensions and individual translator perspective. Chapters explore the
affordances of technological advancements and availability of new tools
in such areas as post-edited machine translation (PEMT) in expanding the
boundaries of what we think of when we think of literary translation,
looking to examples from developments in co-translation, collaborative
translation, crowd-sourced translation and fan translation.
As the first book of its kind dedicated to the contribution CALT in its
various forms can add to existing and future scholarship, this volume will
be of interest to students and scholars in Translation Studies, especially
those working in literary translation, machine translation and translation
technologies.
Andrew Rothwell is Professor Emeritus of French and Translation Studies

at Swansea University, UK.
Andy Way is Professor in the School of Computing and Deputy Director

of the Adapt Centre at Dublin City University, Ireland.
Roy Youdale is Research Associate in Translation Studies at the University

of Bristol, UK.
Routledge Advances in Translation and Interpreting Studies
Computer-Assisted Literary Translation

Edited by Andrew Rothwell, Andy Way and Roy Youdale
Telecollaboration in Translator Education
Implementing Telecollaborative Learning Modes in Translation Courses
Mariusz Marczak
Appraisal and the Transcreation of Marketing Texts
Persuasion in Chinese and English
Mavis (Nga-Ki) Ho
Translating Home in the Global South
Migration, Belonging, and Language Justice
Edited by Isabel C. Gómez and Marlene Hansen Esplin
A Qualitative Approach to Translation Studies
Spotlighting Translation Problems
Edited by Elisa Calvo and Elena de la Cova
Translation and Big Details
Part-Whole Thinking as Practice and Theory
Jeroen Vandaele
Reframing Western Comics in Translation
Intermediality, Multimodality and Censorship
Nicolas Martinez
Translation and Modernism
The Art of Co-Creation
Emily O. Wittman
Translation Studies in the Philippines
Navigating a Multilingual Archipelago
Edited by Riccardo Moratto and Mary Ann G. Bacolod
For more information about this series, please visit: www.routledge.com

/Routledge-Advances-in-Translation-and-Interpreting-Studies/book-series
/RTS
Computer-Assisted
Literary Translation
Edited by Andrew Rothwell, Andy Way

and Roy Youdale
First published 2024
by Routledge
605 Third Avenue, New York, NY 10158
and by Routledge
4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN
Routledge is an imprint of the Taylor & Francis Group, an informa business
© 2024 selection and editorial matter, Andrew Rothwell, Andy Way, and Roy
Youdale; individual chapters, the contributors
The right of Andrew Rothwell, Andy Way, and Roy Youdale to be identified
as the authors of the editorial material, and of the authors for their individual
chapters, has been asserted in accordance with sections 77 and 78 of the
Copyright, Designs and Patents Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced or
utilised in any form or by any electronic, mechanical, or other means, now
known or hereafter invented, including photocopying and recording, or in
any information storage or retrieval system, without permission in writing
from the publishers.
Trademark notice: Product or corporate names may be trademarks or
registered trademarks, and are used only for identification and explanation
without intent to infringe.
Library of Congress Cataloging-in-Publication Data
Names: Rothwell, Andrew, editor. | Way, Andy, editor. | Youdale,
Roy, editor.
Title: Computer-assisted literary translation / edited by Andrew
Rothwell, Andy Way and Roy Youdale.
Description: New York, NY: Routledge, 2023. | Series: Routledge
advances in translation and interpreting studies | Includes
bibliographical references and index. |
Identifiers: LCCN 2023020051 (print) | LCCN 2023020052
(ebook) | ISBN 9781032413006 (hardback) | ISBN 9781032413013
(paperback) | ISBN 9781003357391 (ebook)
Subjects: LCSH: Translating and interpreting—Technological
innovations. | Literature—Translations--History and criticism. |
LCGFT: Essays.
Classification: LCC PN241 .C577 2023 (print) | LCC PN241
(ebook) | DDC 418/.040285—dc23/eng/20230807
LC record available at https://lccn.loc.gov/2023020051
LC ebook record available at https://lccn.loc.gov/2023020052
ISBN: 978-1-032-41300-6 (hbk)
ISBN: 978-1-032-41301-3 (pbk)
ISBN: 978-1-003-35739-1 (ebk)
DOI: 10.4324/9781003357391
Typeset in Sabon
by Deanta Global Publishing Services, Chennai, India
Contents
Contributors viii
Introduction 1
ANDREW ROTHWELL, ANDY WAY AND ROY YOUDALE
PART 1
The Automated and Post-Edited Machine Translation of
Literature 25
1 Literary-Adapted Machine Translation in a Well-Resourced

Language Pair: Explorations with More Data and
Wider Contexts 27
ANTONIO TORAL, ANDREAS VAN CRANENBURGH, AND TIA NUTTERS
2 ‘I Am a Bit Surprised’: Literary Translation and Post-Editing

Processes Compared 53
WALTRAUD KOLB
3 Mark My Keywords: A Translator-Specific Exploration of

Style in Literary Machine Translation 69
MARION WINTERS AND DOROTHY KENNY
PART 2
Machine Translation Applications in Literary Translation 89
4 MT and CAT: Challenges, Irrelevancies or Opportunities for

Literary Translation? 91
JAMES LUKE HADLEY
vi Contents
5 Retranslating Proust Using CAT, MT and Other Tools 106

ANDREW ROTHWELL
6 Author-Tailored Neural Machine Translation Systems for

Literary Works 126
ANTONI OLIVER
7 Machine Translation of Chinese Fantasy (Xianxia) Novels:

An Investigation Into the Leading Websites Translating
Chinese Internet Literature Into English 142
SHUYIN ZHANG
8 Up and About, or Betwixt and Between?: The Poetry of a

Translation Machine 158
TIM VAN DE CRUYS
9 Metaphor in Literary Machine Translation: Style, Creativity

and Literariness 173
ALETTA G. DORST
PART 3
Corpus Linguistics, Text-Visualisation and
Literary Translation 187
10 KonText in Trilingual Studies—Supporting Phraseology

Translation Based on the EPB Corpus 189
ANGELIKA PELJAK-ŁAPIŃSKA
11 Voyant Tools’ Little Outing: How a Text Reading and

Analysis Environment Can Help Literary Translators 203
LISA HORENBERG
12 (Re)creating Equivalence of Stylistic Effect: A Corpus-Aided

Methodology 219
TEREZA ŠPLÍCHALOVÁ
Contents vii
PART 4
Applying Specialised Electronic Tools to
Literary Translation 235
13 The Experiment 237

AVRAHAM J. ROOS
14 Augmenting and Informing the Translation Process through

Workflow-Enabled CALT Tools 258
SASHA MILE RUDAN, EUGENIA KELBERT, LAZAR KOVACEVIC,
MATTHEW REYNOLDS, AND SINISHA RUDAN
Index282
Contributors
Editors
Andrew Rothwell is Professor Emeritus of French and Translation Studies
at Swansea University, UK. He has research interests in contemporary
French literature, especially the poet and art writer Bernard Noël, trans-
lation technologies and translation theory. He has published numerous
literary translations into English, including poetry by Bernard Noël and
two novels by Emile Zola, and is currently working on a new transla-
tion of Proust’s La Prisonnière.
Andy Way is Professor of Computing at Dublin City University, Ireland, and
Deputy Director of the ADAPT Centre for Digital Content Technology.
He is currently writing books on Sign Language Machine Translation
(MT) and European Language Equality arising from projects he coor-
dinates and a textbook on Translation and Automation. He has pub-
lished over 400 papers, was editor of the Machine Translation journal
from 2007–21, and in 2019 received the Award of Honour from the
International Association for Machine Translation for his services to
the community.
Roy Youdale is Research Associate in Translation Studies at the University
of Bristol, UK, where his research interests include corpus stylistics, text
visualisation, and using Computer-Assisted Translation (CAT) tools
for literary translation. He is the author of Using Computers in the
Translation of Literary Style: Challenges and Opportunities. He is also
a literary translator from Spanish to English.
Contributors
Aletta G. (Lettie) Dorst is Senior Lecturer in Translation Studies and English
Linguistics at Leiden University Centre for Linguistics, the Netherlands.
Her research focuses primarily on the translation of metaphor, style
and gender between different languages and cultures, between differ-
ent genres and domains, and between different modes and modalities.
Contributors ix
She is currently leading a Netherlands initiative for Education (NRO)

Comenius Senior Fellow project on the value of machine translation in
Dutch higher education.
James Luke Hadley is Trinity College Dublin’s Ussher Assistant Professor
in Literary Translation, Director of the College’s MPhil in Literary
Translation, and PI of the Irish Research Council-funded QuantiQual
Project, Ireland. His research is representative of his wide-ranging
interests, many of which centre on translation in under-researched cul-
tural contexts, particularly in East Asia. He is also active in machine
translation and computer-assisted translation research, and in inte-
grating Digital Humanities methodologies and empirical research into
Translation Studies.
Lisa Horenberg obtained her MA in Theory and Practice of Translation
from Utrecht University, the Netherlands, in 2019, specialising in liter-
ary translation and translation technology. Additionally, Lisa gained
significant experience by working in the professional translation indus-
try for three years, where she was involved in client and quality manage-
ment, innovation, and the implementation of machine translation.
Eugenia Kelbert is Assistant Professor of Comparative Literature at the
HSE University, Russia, and Early Career Fellow at the University
of East Anglia, UK, following a position at the University of Passau,
Germany, and guest stays at Uppsala and Stockholm universities. Her
dissertation on translingualism (Yale University, 2015) won the ACLA
Charles Bernheimer Prize; she is currently reworking it as a book. She
works on translingual literature, translation, comparative stylistics,
poetry and quantitative literary analysis.
Dorothy Kenny is Full Professor and Chair of Translation Studies
at Dublin City University, Ireland, and a founding member of the
University’s Centre for Translation and Textual Studies. Her recent
publications include the edited volume Human Issues in Translation
Technology and contributions on machine translation and translator
training in The Routledge Handbook of Translation and Philosophy
and The Routledge Handbook of Translation and Technology. She is
the Co-General Editor of Translation Spaces and in 2020 co-edited a
special issue of the journal on Fair Machine Translation.
Waltraud Kolb is Assistant Professor of Literary Translation at the Center
for Translation Studies at the University of Vienna, Austria. She studied
translation (English, French, Portuguese and German) and holds a PhD
in Comparative Literature. One recent focus of her research is on digital
tools and machine translation in the literary field and literary transla-
tion and post-editing processes. She is also a professional translator and
x Contributors
a member of the executive board of the Austrian Association of Literary

Translators.
Lazar Kovacevic is an independent researcher with a focus on the appli-
cation of technology to education, creativity, collaboration and social
action among others. He has done projects in the areas of (web) infor-
mation retrieval systems, text analysis and natural language processing,
machine learning, data mining, and collaboration. He has co-authored
papers discussing creative features in time series ranging from physical
and biological to physiological and psychological processes. He devel-
oped algorithms for increasing the diversity of perspectives in search
results.
Tia Nutters is Lecturer in Translation Studies at Utrecht University and at
the University of Groningen (UG), the Netherlands, and grant proposal
editor at the UG. Her research interests mainly concern the application
of translation theory in practice as well as the application of Machine
Translation (MT) in the literary translation process.
Antoni Oliver (Barcelona, 1969) holds a PhD in Linguistics, a degree in
Slavonic Languages and Literature from the University of Barcelona,
Spain, a degree in Telecommunications Engineering from the Polytechnic
University of Catalonia, Spain, and a Masters Degree in Free Software
from the Open University of Catalonia. He is an associate professor
at the Open University of Catalonia and the director of the MA in
Translation and Technologies. His main area of research is Natural
Language Processing, with a special focus on machine translation and
automatic terminology extraction.
Angelika Peljak-Łapińska is a corpus linguist at the University of Lodz,
Poland. She is currently compiling the spoken corpus of Polish within
the EU-funded CLARIN-PL project ‘Common Language Resources
and Technology Infrastructure’. Angelika is a graduate of Swansea
University, UK, and the University of Warsaw, Poland. Her main aca-
demic interests are corpus linguistics in Translation Studies and dis-
course analysis in English, Polish and Belarusian.
Matthew Reynolds is Professor of English and Comparative Criticism
at the University of Oxford, UK, where he chairs the Oxford
Comparative Criticism and Translation Research Centre (OCCT)
and leads the Prismatic Translation project. Among his books
are Prismatic Translation, Translation: A Very Short Introduction, The
Poetry of Translation: From Chaucer & Petrarch to Homer &
Logue, Likenesses , The Realms of Verse: English Poetry in a Time of
Nation-Building, and the novels Designs for a Happy Home and The
World Was All Before Them.
Contributors xi
Avraham J. Roos did his PhD studies at the Amsterdam School of Historical
Studies, which is part of the University of Amsterdam, the Netherlands.
Roos did research comparing 106 English variant retranslations of the
Hebrew Passover Haggadah―a semi-liturgical work recited by Jews on
Passover Eve―using both algorithmically enhanced semi-close-reading
techniques and regular close reading. His goal was to visualise the differ-
ences by means of digital tools and to explain these differences against a
cultural-historical backdrop. Roos lectures at Herzog Academic College
and Orot Yisrael College in Israel.
Sasha Mile Rudan is completing his PhD at the Department of Informatics,
University of Oslo, Norway, on collaborative face-to-virtual systems for
augmenting social processes, knowledge management and dialogue. He
is a researcher at the Department of Linguistics and Philology, Uppsala
University, Sweden, on infrastructure for researching hagiographic
texts. He founded and co-leads the LitTerra Foundation which digitally
supports literature and cultural heritage. It provides digital humanities
(DH) platforms for computational text analysis (http://litterra.net/buk-
vik) and visualisation (http://litterra.net/litterra).
Sinisa Rudan is а researcher, IT developer and poetry performer. His
research is in the domain of collaboration and creativity, applied
on socio-technical ecosystems for social good/activism. He leads sev-
eral art-science multidisciplinary projects. He pursues his interests
through several regional and international positions: Co-founder of
‘ReMaking Tesla – Practices That Make a Genius’―International
Forum of Interactive and IT-Augmented Education; Co-founder
of ChaOS―an NGO uniting artists and scientists on cultural and
humane projects.
Tereza Šplíchalová is a freelance translator and a PhD student in English-
Czech Translation Studies at Masaryk University, Czech Republic. Her
research revolves around the interdisciplinarity of translation studies,
be it in relation to narratology, applied linguistics or logic and currently
focuses on the interface between Translation Studies and fictional world
theories.
Antonio Toral is Assistant Professor in Language Technology at
the University of Groningen, the Netherlands. He holds a PhD in
Computational Linguistics from the University of Alicante, Spain, and
has been researching in the field of Machine Translation (MT) since
2010. His research interests include the application of MT to literary
texts, MT for under-resourced languages, and the analysis of transla-
tions produced by machines and humans.
xii Contributors
Andreas Van Cranenburgh is Assistant Professor at the University of

Groningen, the Netherlands, working on Computational Linguistics
and Digital Literary Stylistics. His dissertation and current research are
on the automated analysis of syntax and its applications to literary text
analysis.
Tim Van de Cruys is an associate professor within the Linguistics
Department at the Faculty of Arts, KU Leuven. His main research inter-
est is natural language processing, with a particular focus on the unsu-
pervised modelling of meaning, the analysis of multivariate language
data within the mathematical framework of tensor algebra, and creative
language generation.
Marion Winters is Senior Lecturer in the Department of Languages and
Intercultural Studies at Heriot-Watt University, Scotland. She is the
founding editor of New Voices in Translation Studies, a member of
the IATIS Publications Committee and a professional member of the
German and Irish translators’ associations (BDÜ, ITIA). She has pub-
lished widely on translator style and translated autobiographical writ-
ing and is co-editor of the book Translating Holocaust Lives. In her
current research project, in collaboration with Dorothy Kenny and
the literary translator Hans-Christian Oeser, she investigates how
the increasing use of machine translation may be affecting literary
translators.
Shuyin Zhang is Lecturer of Translation Studies in the School of Humanities
and Social Science at The Chinese University of Hong Kong, China,
where she teaches culture and translation, research in translation prac-
tice, and translation project management at postgraduate and under-
graduate levels. She holds a PhD in Translation Studies and an MA in
Linguistics from University College London, UK. Her research interests
focus on machine translation of Chinese fantasy literature, metaphor in
classical Chinese literature translation, and translation and paratexts.
Introduction
Andrew Rothwell, Andy Way and
Roy Youdale
I.1 Computer-Assisted Literary Translation (CALT)—the

Background
The practice of literary translation has until quite recently been widely
held, by both translation scholars and literary translators themselves, to
be an ancient and traditional field of translation which neither wants nor
needs the assistance of computer technology, beyond the now ubiquitous
functionalities of word-processing and internet searches (Daems, 2022;
Ruffo, 2022; Way, Rothwell and Youdale, 2023). Such attitudes still
persist amongst a minority of literary translators, and it is the aim of this
collection to challenge such views and to provide evidence for the idea that
while certain aspects of technology can be argued to have the potential
to negatively affect the work of literary translators, this is by no means
true of all translation technologies, which collectively offer a range of
creative opportunities to enhance their work. The collection is aimed at
several groups of potential readers: Translation Studies (TS) scholars and
students, literary translators and the developers of translation technologies
and tools.
Before exploring these creative opportunities, in a volume on the topic, it
is appropriate to deal with the question of what literary translation actually
is and if and how it can be distinguished from non-literary translation. It
is argued by Hadley et al. that ‘it is not clear whether there is or could
be a clear divide between literary and non-literary forms of translation’
(2022, p. 4), and the same argument applies to the original texts. We see
attempts to create a hard and fast binary distinction as both unhelpful and
unfounded, but the fact remains that the distinction is used to sustain the
myth that the suitability or otherwise of computerised tools represents a
clear dividing line between literary and non-literary translation, and it is in
this context that we locate our discussion.
When it comes to defining what constitutes a ‘literary’ text, and by
implication a literary translation, there appear to be two broad camps. On
the one hand, there are those who effectively avoid attempting to define
DOI: 10.4324/9781003357391-1
2 Andrew Rothwell et al.
literariness at all by simply referring to ‘literary texts’ and assuming that

everyone will understand what this means; or (which boils down to much
the same thing) saying that literary texts are simply those which people
describe as such. On the other hand, there are translators and translation
scholars who believe that it is possible to make a clear distinction between
literary and non-literary texts, but the basis for this distinction varies.
Landers, for example, argues that ‘More than in other branches of the
translator’s art such as legal, technical/scientific, financial/commercial, or
in interpretation (simultaneous or consecutive), literary translation entails
an unending skein of choices’ (2001, p. 9). Newmark’s view is that ‘The
main intentional difference between literature and non-literature is that
the first comprises the world of the mind and the imagination; the second,
the world of reality, of facts and events’ (2004, p. 10), while Boase-Beier
suggests that
the style of a literary text involves the use of such figures as metaphor,
ambiguity, and repeated patterns. These figures may be present in any
type of text, but they will often be less frequent, less complex and less
subtle in non-literary texts.
(2010, p. 30)
Another variation involves the attempt to make a distinction between

‘creative’ text translation and other types of translation. Hadley et al.
define creative-text translation as
the translation of texts from one language to another where the texts
themselves pivot broadly on the human creativity employed in their
production. They rely heavily on aesthetics for their existence, more
than texts that aim to bring about an outcome directly, as in the case
of technical texts.
(2022, pp. 5–6)
It has also been suggested that there is such a thing as ‘literary language’
which distinguishes literary from non-literary texts (Fabb, 2010). This
proposition we unreservedly reject, since (the same) language is used for
whatever people want to use it for. Literature often uses very simple lan-
guage, as exemplified by Ernest Hemingway for instance (see Section 2.1,
Chapter 2, this volume), while so-called technical/commercial translation
often uses complex and unusual language. Nor do we think that literary
texts can be clearly distinguished by their use of formal linguistic features
such as metaphor (but see Chapter 9, this volume, for a discussion of
metaphors and MT), wordplay etc. since these are employed in a wide
variety of text-types albeit to varying degrees. Toral et al. (Section 1.4,
Introduction 3
Chapter 1, this volume) discuss literariness and its ‘predictability’ based on

the automated detection and analysis of specific linguistic features.
There is, however, one characteristic of texts which are frequently
labelled as literary which does appear to command fairly widespread
recognition, and that is the inseparability of style and content (Landers,
2001; Newmark, 2004; Boase-Beier, 2010). As Youdale comments, ‘Style
is important to literature because it focuses not just on what is said, but
on how it is said, and why a text may be shaped in a particular way’
(2020, p. 2). The literary critic Terry Eagleton argues that literary writing
‘is the kind of writing in which content is inseparable from the language
in which it is represented. Language is constitutive of the reality or expe-
rience, rather than simply a vehicle for it’ (2014, p. 3). The focus of this
collection is on the ways in which translation technologies can be shown
to offer literary translators an augmented variety of ways in which to try
and achieve that goal of retaining both style and content.
Cronin (2010) argued that TS, and by implication literary translation,
underwent a ‘technological turn’ at the end of the 20th and beginning of
the 21st centuries ushered in by the increasingly widespread availability
of personal computers, large databases of machine-readable electronic
texts and the software to process them. However, it pays to contextualise
the impact of 21st-century computer technology on literary translation
by briefly considering the relationship between technology—particularly
writing technology—and translation over a much longer period.
Littau (2011) argues that since practices of reading and writing have
varied over time in accordance with the physical media available for stor-
age and retrieval, the practice of translation, which very often involves
reading and writing, has also been heavily influenced by them. Two short
examples used by Littau make this clear: (i) with the invention of mov-
able metal type-printing in Europe between 1445 and 1450, driven by the
Bible, came the ability to produce books on an unprecedented scale and
reach a mass audience.1 This created a big demand for translations, since
most people were unable to read Latin or Greek, and wanted to be able to
read in their own language, something which inspired Luther’s translation
of the Bible into German, and (ii) around 1860―over 1,000 years after its
invention in East Asia (Yum, 2011)―the ‘invention’ of pulp allowed paper
to be made cheaply from wood instead of using rags and stimulated the
demand for translation since popular works from one country were quickly
translated for readers in other countries. Thus, a new technology directly
influenced the quantity, style and speed of interlingual literary translation
in parts of the Western world. These examples support the view that at
1 It should be noted that by the 15th century movable type had already been in use in East
Asia for at least two centuries (Brokaw & Kornicki, 2016).
any given point in time, translation—including literary translation—is very

much ‘the product of a particular technology’ (Littau, 2011, p. 266), and
that emerging technology or technologies have always helped to determine
what is translated, where, when, how, by whom and for whom.
Over the past five to ten years, a number of factors have combined to
make inroads into the near universal rejection of the idea that computerised
translation technologies could be of use in literary translation (Daems,
2022; Ruffo, 2022; Way, Rothwell and Youdale, 2023). Not only has the
quality of machine translation (MT) output improved significantly and
computer-assisted translation (CAT) tools become more sophisticated
and user-friendly, but a new generation of literary translators seems at
least willing to explore their use in various ways (Slessor, 2020). While
the achievement of fully automated high-quality literary translation of
complete works seems almost as far off as ever, the use of post-edited
MT (PEMT), particularly within the context of the convergence of MT
with translation memories in CAT tools, offers a more realistic prospect.
This use of MT also fits in well with the growing world of fan translation
(see Chapter 7, this volume) and projects such as bilingual e-books (see
Chapter 6) aimed at language students, where only a ‘good-enough-to-
understand-meaning’ translation is required, rather than a professional
human-quality literary version.
Many of the features which make CAT tools advantageous for
commercial translators—such as ease of finding your place in the text,
version retention and concordance searchability of both source and target
texts—have been argued to work just as well for their literary counterparts
(Rothwell and Youdale, 2022, and Chapter 5, this volume). In a similar
vein, corpus linguistics tools and text-visualisation programs can be used
by translators to analyse the source text (ST) before and during translation
(see Chapters 11 and 12, this volume) to analyse a draft translation at the
revision stage and to undertake an analysis of their own translator style
(Youdale, 2020).
Over the last ten years there have been individual projects and articles
on the application of IT to literary translation, such as Lee (2011), but as
a term describing a specific field practice within TS, CALT has only taken
shape recently. Formally, it can be argued to date from the titles of two
international CALT workshops hosted by Swansea University in 2019. It is
a field still in its infancy, yet to be acknowledged in mainstream TS. In the
Routledge Handbook of Translation and Technology (O’Hagan, 2020)
the word ‘literary’ still does not appear in the index or in the titles of any
of the 31 chapters in the volume. However, the existence of a field of study
can be evidenced by reference to events and publications. To date there
have been seven international workshops and conferences dealing with
topics falling under the CALT umbrella: the January and May 2019 CALT
Introduction 5
workshops at Swansea University2; the August 2019 Dublin Machine

Translation Summit: The Qualities of Literary Machine Translation3;
the panel on Translation Technologies for Creative-Text Translation at
the September 2019 European Society for Translation Studies conference
in South Africa4; the May 2021 three-day CALT2021@Swansea online
conference hosted by Swansea University5; the Creative Texts, Technology
and Ecology panel at the International Association for Translation and
Intercultural Studies (IATIS) conference in Barcelona in September
2021;6 and the November 2021 ASLING Translating and the Computer
Conference 43.7 All the chapters in this collection are based on papers
presented at these events, and the book attempts to take stock of early lines
of enquiry in this developing field.
In terms of publications, there is as yet no specific journal or online
forum dedicated to CALT. However, in the past five or six years a number
of studies, mainly on MT and literary translation, have appeared in
various journals or as book chapters, and one full-length book has been
published on the computer-assisted translation of literary style (Youdale,
2020). An edited collection has been published (Hadley et al., 2022) on
the use of translation technologies in the translation of ‘creative’ texts,
encompassing literary texts but also extending beyond them to cultural
texts more broadly.
Another way in which an academic field can be defined can be found
in the kind of research questions which it pursues and for CALT these
include, for example:
• How does CALT affect the nature of literary translation as an activity

and as a product?
• To what extent can PEMT be seen as potentially enabling the production
of literary translations which would otherwise not be done at all (see
Chapters 6 and 7, this volume)?
• How do the various technologies work with different language pairs
and directions?
• How can electronic tools be used to enhance the work of literary
translators?
2 https://ilcs.sas.ac.uk/node/12053.
3 https://www.academia.edu/38505151/Machine_Translation_Summit_2019_Workshop
_Call_for_Papers_The_Qualities_of_Literary_Machine_Translation.
4 https://www.academia.edu/38087247/Call_for_papers_Translation_technologies_for_cre-
ative_text_translation.
5 https://calt2021conference.wordpress.com/.
6 https://www.iatis.org/index.php/7th-conference-barcelona-2021/item/2242-panels#P13.
7 https://www.asling.org/tc43/.
• How can these technologies be used in the translation of different lit-

erary genres and different linguistic features such as metaphor (see
Chapter 9, this volume) and wordplay?
• What kinds of ethical and legal issues are raised by the use of translation
technologies (see Moniz and Escartín, 2023)?
• What are the implications of CALT for the training of literary translators
and for the evaluation of literary translations?
• How do literary translators respond to these technologies and what do
they see as the threats and opportunities created by them (see Chapter 4,
this volume, and Way, Rothwell and Youdale, 2023)?
We believe that this is the first book to bring together a collection of studies
which both illustrates the range and depth of research being undertaken in
CALT and assesses its development and future potential. The traditional
model of a single translator producing a translation of a literary work,
while probably still the norm, has been supplemented by other ways of
working including co-translation, collaborative translation where members
of a group share their drafts for comment and criticism and crowd-sourced
and fan translation where non-professional enthusiasts post their versions
of genre fiction on the internet (see Chapter 7, this volume). And it is the
availability of new electronic tools that makes all this possible.
What unites all the work displayed in this collection is a focus on the
application of theory to the practice of literary translation. In contrast
to the underlying philosophy of Descriptive Translation Studies (Toury,
2012)—that the job of TS is not to prescribe how translation should be
done, but to describe how it is done—CALT is founded on the idea that
the possibilities held out by translation technologies, and technologies
that can be used for translation, should be explored by both scholars and
practitioners of the art. In contrast to corpus-based Translation Studies,
it is concerned with how technology can be used in the process of literary
translation, and not simply in the analysis of already published translations,
valuable though that is.
I.2 Machine Translation—the State of the Art

From the early 1990s to the mid-2010s, statistical MT (SMT: Koehn et al.,
2003, 2007) provided the best MT quality. However, the emergence of the
neural paradigm quickly changed this. Way (2020, p. 316) notes
that ‘while the preferred system set-ups were not so different from
those that had been conceived some time before (e.g. Forcada and
Ñeco, 1997), the hardware that facilitated the huge explosion in com-
putation required was now sufficiently powerful to allow these sys-
tems to be built in practice.’
Introduction 7
Accordingly, once competitive neural network (NN)-based approaches to

MT appeared (Cho et al., 2014), there was no looking back, and from
2016 the neural approach became the preferred method for automatic
translation systems in almost all use-cases (cf. Bentivogli et al., 2016;
Castilho et al., 2017).
Van de Cruys (Chapter 8, this volume, Figure 8.1) demonstrates how
NNs comprise an input layer, an output layer and at least one hidden
layer in between (made up of a number of neurons, where all neurons
in one layer are connected to all neurons in the previous and subsequent
layer by weighted arcs). Initial attempts like that of Cho et al. (2014) to
use NNs in translation showed the performance to deteriorate on longer
strings, owing to source sentences being encoded as fixed-length vectors
(i.e. geometric objects having length and direction in multidimensional
space). Van de Cruys (Section 8.3, Chapter 8, this volume) explains how
recurrent NNs (RNNs) can be used by replacing fixed-length vectors with
variable-length vectors (cf. Bahdanau et al., 2015). In an encoder-decoder
setup (see Van de Cruys, Section 8.5, Chapter 8, this volume), RNN mod-
els enable previous outputs to be used as inputs through the use of hidden
states. The source sentence in its entirety is processed by the encoder, and
the hidden state of the network after the whole sentence has been pro-
cessed is passed to the decoder to generate the ‘best’ translation it can.
In a nutshell, it is this ability to encode the entire input string in one rep-
resentation that gives NMT its principal advantage over SMT, where a
source sentence is only translated using lexical and phrasal partial chunks.
This final encoder state is obtained after many forward and backward
passes through the NN, with each forwards-backwards pair known as an
‘epoch’. On the forward pass, an MT hypothesis is output by the system
for a sentence in the development set (see below), which is compared to
an ideal ‘reference’ translation (see below), and errors back-propagated
through the network and the weights on the arcs updated. On the next
pass, hopefully the new hypothesised translation is better than the one
before in terms of an automatic evaluation score (see below), and the pro-
cess continues until no further improvements can be made (or an ‘early
stopping’ criterion is invoked, i.e. no (significant) increase in terms of an
automatic score for n epochs, see Oliver, Section 6.4.4, Chapter 6, and
Toral et al., Section 1.2.4, Chapter 1, both this volume)8 At this point,
8 NMT training is computationally expensive, so some decisions are taken which although
unprincipled from a linguistic or translational perspective are valid computationally. For
instance, the vocabulary is usually limited which results in some bona fide words of either
source or target language being unknown to the system. In order to try to process such
out-of-vocabulary (OOV) items in terms of similar looking words that actually are con-
tained in the training data, prior to training and to translating the data is segmented into
the NMT system is frozen as the final model and used to tackle unseen
sentences from the test set (see below).
More importantly, Bahdanau et al. (2015) introduced the concept of
‘attention’ to the basic RNN architecture, similar in spirit and intention to
source–target ‘alignments’ in SMT. The attention mechanism helped the
decoder choose the most appropriate target word for a particular source
item by allowing it to consider the most relevant other source-sentence
words to use in translation as contextual clues for translationally ambigu-
ous words in the same string.
Vaswani et al. (2017) took this a step further by using attention as
‘all you need’ to translate in their ‘Transformer’ architecture (cf. Van de
Cruys, Section 8.2.4, Chapter 8, this volume). Transformer-based systems
have achieved state-of-the-art (SOTA) performance on many benchmarks
in language processing, so it is unsurprising that Toral et al., Oliver and
Van de Cruys all use such systems for their MT experiments in this volume.
Before we describe how MT was used for literary translation elsewhere,
it is worth explaining the three different phases of building, tuning and
evaluating corpus-based (SMT and NMT) systems. Traditionally, large
amounts of high-quality parallel data were needed to build such systems
(and for SMT, large amounts of monolingual data, too). This comprised
source-language sentences and their human translations, such as the
proceedings of the European Parliament that have been used for MT engine-
training for many years (Koehn, 2005). For most official EU language-
pairs, around 2 million parallel sentence pairs exist in this collection,
which is usually enough to build good-quality MT systems, certainly as
far as SMT is concerned.9 Most of this material would be used for MT
system building (the ‘training’ set), but small subsets of it would be used as
‘held-out’ data for tuning the parameters of the system (the ‘development’
set) and a separate set for evaluating the quality of the tuned system (the
‘test’ set). Note that both the development and test sets are held out from
the training set so as not to unfairly skew (‘bias’) the performance of the
system.
subword units (not necessarily morphemes in the linguistic sense) during pre-processing.
Post-processing after translation consists of merging subword units into full wordforms
(e.g. “straight ##forward” becomes “straightforward”).
9 Khayrallah and Koehn (2018) demonstrate that with the same amount of training data
NMT models underperform compared to SMT, but with much larger corpora they can
outperform such systems. Accordingly, lots of work has gone into generating synthetic
(MT-generated) data via back-translation (Sennrich et al., 2016; Poncelas et al., 2018),
as larger amounts of authentic (human-produced) data are simply unavailable. Despite
the fact that back-translated data might be considered noisy, NMT systems trained with
such additional data typically outperform those trained solely on human-generated ‘gold-
standard’ corpora (cf. Chapter 1, this volume).
Introduction 9
In these latter two phases, the human translation is deemed to be ‘the’

correct translation, against which the MT output is compared using a
range of automatic evaluation metrics. Way (2018) describes the relative
merits of using human experts and automatic methods to determine the
quality of the MT outputs, but essentially automatic metrics are used as
human experts are not always available. Automatic metrics are broken
down into precision-based measures like BLEU (Papineni et al., 2002) and
chrF (Popović, 2015), where a higher score is better, or error-based metrics
like TER (Snover et al., 2006) where a lower score indicates better quality.
Finally, returning to how MT systems are built (see Forcada, 2017 and
Way, 2020 for more detail on all these components, written for a lay audi-
ence, or Koehn, 2020 for a more in-depth technical explanation and dis-
cussion), large language models (LLMs: collections of words and phrases
of a language, usually organised by length; hence unigrams (individual
words), bigrams (pairs of words, not necessarily phrases in the linguistic
sense) and trigrams (3-word sequences) etc.) such as OpenAI’s ChatGPT
(Ouyang et al., 2022) are currently attracting great interest. Way (forth-
coming, 2024) laments yet again the huge hype associated with their
development, especially the claim of Van der Meer (2021) that human
translators are doomed as their jobs will be taken over completely by
machines. Nonetheless, their potential is there for all to see. Van de Cruys
and Roos (Chapters 8 and 13, this volume) give some thought to how such
technology can be applied to poetry translation, and while LLMs may cur-
rently underperform compared to bespoke NMT systems such as those
built using the Transformer architecture, it will not be long before they
start to demonstrate improved translation quality, as well as better perfor-
mance on a host of NLP tasks, especially if combined with some human
hand-holding (Roos, Chapter 13, this volume).
I.3 Machine Translation of Literature—the State of the Art

Even prior to the emergence of NMT, the current state of the art, SMT
performance was good enough to be regularly used by a significant number
of users across a range of application areas. Way (2018, 2020) provides a
couple of examples of MT usage at Google and Facebook, showing that
just for Google Translate10 more than 5 years ago, almost 150 billion
words were being translated on a daily basis.11
10 https://translate.google.com.
11 Bringing this more up to date, Trancoso (2022) notes that ‘the volume of words that are
daily translated using publicly available machine translation engines reaches several tril-
lion words’, citing a 3-year-old SDL blog written by K. Vashee: https://www.rws.com/blog
/The-Issue-of-Data-Security-and-Machine-Translation/.
In the mid-2010s, Way and Toral chose as a research topic what they
thought was the ‘last bastion of human translation’, namely the translation
of literature, as ‘the perceived wisdom is that MT [wa]s of no use for
the translation of literature’ (Toral and Way, 2015a, p. 123). Despite the
impressive improvements in MT that had been seen by that time, neither
Toral nor Way really believed that good translation quality was possible
in what is perhaps the most creative task a human translator can take on,
using an MT system.
They soon discovered that there was, in fact, very little related work
on the topic. MT had been used for some literary translation experiments;
for example, both Genzel et al. (2010) and Greene et al. (2010) examined
the use of SMT for poetry, but the former article did not test their French–
English system on poems (just news texts), while the latter evaluated the
MT output only from a qualitative perspective. Voigt and Jurafsky (2012)
examined the topic of referential cohesion of French prose and poetry
translated into English. Closer to what they planned to do was the work of
Besacier (2014), where PEMT (performed by non-professional translators)
of a short story was used to translate a short story from English into French.
However, they could not find any previous work where MT had been
applied to novels. In the data-gathering phase with which any MT developer
is familiar, they found quite a lot of literary parallel data available in the
form of e-books. Initially, then, in Toral and Way (2015a), they built a
tailored SMT system for a contemporary best-selling author, Carlos Ruiz
Zafón, and then used it to translate one of his novels—El Prisionero del
Cielo [The Prisoner of Heaven] (2011)—between two closely-related
languages, namely Spanish to Catalan; if the MT quality was too poor
between languages with such similarity, they could abandon their venture
and declare the bastion to be impregnable!
In fact, far from being poor, their results were surprisingly good. Not
only was the performance of their system better than three other systems
(including Google Translate) according to three standard automatic metrics,
but for 20% of the sentences, the translations produced by the MT system
and a professional translator (i.e. those taken from the published novel
in the target language) were identical. In addition, a human evaluation
revealed that for over 60% of the sentences, native speakers deemed the
translations of the MT system to be of the same quality as those of the
professional translator.
Duly encouraged by these findings they decided to move on to a less
closely related language pair (English to Catalan) and to extend their
experiments with the newly emerging NMT paradigm which at the time
was based on RNNs with attention (Bahdanau et al., 2015). In Toral and
Way (2018), they compared the performance of (at the time) large-scale
models of SMT and NMT, using 133 parallel novels as training data—a
Introduction 11
total of over 100 million words of literary text—and 1,000 novels in

Catalan, which were used to train the SMT target language model, as well
as additional synthetic training data for the NMT system following back-
translation (see note 9, above). They performed automatic evaluation on
12 very well-known books across a range of genres from the 1920s to the
current day, and a human evaluation on three of them. They demonstrated
that on average, NMT outperformed SMT by about 11% relative, according
to the BLEU automatic evaluation metric, although as might be expected,
the quality varied significantly depending on the novel to be translated.
The human evaluation again used professional translators (as did the post-
editing), but as an improvement on the sentence-based evaluation in Toral
and Way (2015a, 2015b), and informed by their investigations (Toral
et al., 2018) on Microsoft’s ‘human parity’ paper (Hassan et al., 2018),
they extracted sets of contiguous sentences in order to examine document-
level translation features. The human evaluation showed that depending
on the book, between 8% and 20% of the translations produced by the
SMT—rising to between 17% and 34% for NMT—are perceived by native
speakers of the target language to be of equivalent quality to translations
produced by a professional human translator.
In a continuation of that work, Moorkens et al. (2018) performed
an experiment with six professional translators who were asked to do
an English-to-Catalan literary translation under three conditions: (i)
translation from scratch, (ii) SMT post-editing and (iii) NMT post-editing.
Despite a clear improvement in productivity when post-editing NMT, the
translators still preferred to translate from scratch, as they felt there were
fewer constraints on their creativity. When comparing the two types of
MT systems, the translators saw an improvement in terms of the quality
of the output from NMT, which also required fewer edit operations to
transform the output into an acceptable translation. These findings were
further examined by Toral et al. (2018) in measuring (i) the number of
keystrokes (the technical dimension), (ii) the time taken to translate each
sentence (temporal dimension) and (iii) the number of pauses made as well
as their duration (cognitive effort). Both MT approaches led to an increase
in productivity, reductions in the number of keystrokes and also cognitive
effort. However, there were longer pauses when MT was used, and NMT
performance was affected by the length of the sentence being translated.
Another relevant topic in the scope of translating literature is that of
‘translationese’, where the MT system produces language that when post-
edited would not necessarily match that of human professionals generating
translations from scratch.12 Two papers that investigated this topic came
12 Note that some research on translationese purely in human translation contexts exists,
e.g. Meldrum (2009).
out independently at the same time, appearing at MT Summit 2019, where

Toral (2019) won the best paper award, and Vanmassenhove et al. (2019)
was runner-up. Toral and Way have described how previous work focused
on relatively crude measures such as the number of errors. Here Toral
examines PEMT and human translation via three of Baker’s (1993) ‘trans-
lation universals and laws of translation: simplification, normalisation and
interference’ (Toral 2019, p. 273), for a range of language pairs and text
types. He finds evidence of ‘post-editese’ which he notes to be lexically
simpler than translationese. He also demonstrates that the length of post-
edited MT sentences is more similar to the length of the input string com-
pared to translations produced from scratch, and that part-of-speech (PoS)
sequences in PEMT-ed outputs are more similar to PoS sequences in the
source language than typical human-generated translations. Given these
findings, one can expect qualitative differences in the end-user’s experi-
ence of reading a post-edited novel compared to reading that novel having
been translated by a professional translator. Vanmassenhove et al. (2019)
observe similar target-language impoverishment in their study. They show
for two language pairs how SMT and three different types of NMT sys-
tems are unable to render the lexical diversity of translations produced
by humans. Their hypothesis is that the MT systems tend to ‘exacerbate
already frequent patterns while ignoring less frequent ones’ (p. 222). In
a similar vein, Webster et al. (2020) compare the performance of Google
Translate and DeepL13 when translating four classic novels from English
into Dutch. Similar to previous work, a lower level of lexical richness and
local cohesion was observed in the NMT output compared to the human
translations, and there were more similarities between the MT output and
the syntactic structure of the source sentences.
One can see that all these findings—extended in Vanmassenhove et al.
(2021)—might also impinge on the experience of the reader; there are text
types (such as software manuals) where repetition and restricted vocabulary
are beneficial, but surely this is not the case in literary translation? This is
examined specifically by Guerberof-Arenas and Toral (2019), who analyse
how engaged readers feel when experiencing fiction translated from English
to Catalan under three conditions: from scratch (fully human translation),
machine translation and PEMT. As expected, readers were more engaged
with the human-translated text, but it was surprising that they preferred
the machine-translated text over the post-edited output.
Other papers also investigate the characteristics of translations pro-
duced by PEMT compared to translating from scratch. Farrell (2018) finds
that for a range of online English-to-Italian MT systems, post-edited MT
13 https://www.deepl.com/en/translator.
Introduction 13
outputs contain ‘markers’ that signify that their provenance is MT. He

concludes that ‘failure to eradicate these markers may eventually lead to
lexical impoverishment of the target language’. In Chapter 3 of the present
collection, Kenny and Winters demonstrate using corpus linguistic tech-
niques that the style of a highly experienced English-to-German literary
translator changes when using MT.
As a sign that more authors have been persuaded that using MT to
translate literature may be beneficial, a number of papers exploring the
idea for new language pairs have recently appeared. Tezcan et al. (2019)
examine the performance of Google Translate on English-to-Dutch liter-
ary translation using three criteria: lexical richness, cohesion and syntactic
equivalence. As in other work reported above, they, too, find stylistic dif-
ferences between human and neural translation. Matusov (2019) builds
tailored NMT systems for translating literature from English to Russian
and German to English—although he uses different NMT architectures
for each language pair—and includes a new error classification schema
designed specifically for literary MT. Kuzman et al. (2019) compare a range
of bespoke literary NMT systems against Google Translate for English to
Slovene. Despite not being able to outperform Google Translate, trans-
lation productivity did increase for the better bespoke systems in post-
editing tasks for a number of student participants, but not by a significant
amount.
Toral et al. (2020) bring the NMT work right up to date using the
SOTA Transformer architecture (Vaswani et al., 2017) which was not
available when Toral and Way did their formative NMT experiments.
Compared to the SMT and RNN-based NMT systems that Toral and Way
(2018) used, and working on the same data as they used in that study, the
state-of-the-art Transformer model considerably outperformed the other
two systems.14
I.4 Non-MT CALT—Research Directions

If the impact of raw or post-edited MT on literary translation may be
characterised as broadening, potentially making available to readers
around the world more works of literature translated from and into more
languages, other computerised technologies such as those discussed in this
14 Note that the work Toral and Way described in this section either uses MT systems built
by the authors themselves, or freely available MT such as Google Translate. Of course,
where systems have been built which are outside the control of those testing their quality,
there is no guarantee that the test data used in these studies is not contained in the train-
ing data of those freely online MT systems, which could have skewed the results. Clearly
where authors were able to build their own systems, this can be avoided.
book serve to deepen the translator’s relationship with both the ST and
the TL (target language) into which it is being translated, empowering
them to make use of the added information without applying it through
any automated process. In recent years, literary translators have begun
to appropriate a range of tools primarily developed for use in corpus
linguistics and text visualisation for their own purposes, not just to compare
features of existing translations (see especially Cheesman et al., 2017), but
to inform and guide their approach to performing new ones. Different
types of computer-enabled analysis applied to the literary ST, treated
as (part of) an electronic corpus, can yield a wealth of insights into its
thematic and lexical structures which go far beyond what can typically be
derived from a conventional linear reading, however detailed. As Youdale
(2020) has shown, both close and distant (Moretti, 2013) corpus analyses
can reveal specific features of an ST author’s style, either in general or in
a particular work (see also Chapter 3, this volume) which the translator
can then decide whether to try to emulate in the TT. Data visualisation
(see also Chapters 11 and 14, this volume) can make such decisions clearer
by showing in graphical form, for instance, where in the ST a particular
theme is prominent, how patterns of imagery cluster around a particular
character, or where the length of sentences departs from the average,
raising the question of the author’s literary objective in doing these things.
While the specific use (if any) that the translator makes of this additional
rich information remains entirely a matter for them, and the analysis tools
from which it derives in no way dictate or determine the decisions they will
subsequently make, the result of using them in a preparatory way can be a
better-informed, stylistically aware translation.
From the wealth of imaginative, sometimes ludic applications of
computer technology to literary translation described in the different
chapters of this book (see Part 5 below) it is clear that practitioners and
researchers alike are discovering myriad ways of enriching their analytical
engagement with source texts and multiplying their perspectives on the
production of new target texts. The internet-connected, tools-equipped
literary translator of the 21st century appears to have a much more powerful
relationship with their materials, both cultural and linguistic, than one
working even a couple of decades ago with little more than a book, paper
or CD-ROM dictionaries, and a word processor. Although coined with
‘human-in-the-loop’, MT-derived commercial translation in mind, the
concept of the augmented translator (Lommel, 2018) now seems at least as
applicable, if not more so, to state-of-the-art literary translation practice.
While new applications specifically designed for the literary domain are
being prototyped (custom-trained MT systems, translation comparison
and analysis tools such as KonText and Bukvik, see Chapters 10 and 14,
this volume), others are being experimentally appropriated from fields
Introduction 15
such as NLP and text visualisation. This is especially the case with the
kind of interactive environment offered by commercial CAT tools which
allow the ST and developing TT (target text) to be presented on screen in
parallel alignment, avoiding the need for the translator’s gaze to flit back
and forth between page and screen with the attendant risk of omission
while subsequently facilitating revision of the draft TT. Many CAT tools
additionally allow the integration of MT into the workbench environment,
giving the translator instant access to a range of relevant lexical (including
collocational) options, even if rarely to a fully satisfactory translation
proposal (see Chapter 5, this volume). We might speculate that such tools
will eventually be customised to offer the augmented literary translator
of the future an integrated workbench suite giving fine-grained access
to the patterns underlying the ST as well as leverage of a range of TL
linguistic resources, from online dictionaries and thesauri to MT proposals
and, through alignment and translation memory, the decisions made by
previous translators (see Chapter 14, this volume).
I.5 The Chapters

The 14 chapters in this book were originally given as papers at CALT
research events held both in person and online between January 2019 (first
Swansea University CALT Workshop) and November 2021 (ASLING
Translating and the Computer Conference 43). They are divided into four
thematic parts, the first two are devoted to the topic of MT and literature,
the third to the use of corpus-linguistic and text visualisation techniques
in literary translation, and the last to the application of other specialised
electronic tools.
Part 1, on The Automated and Post-Edited Machine Translation
of Literature, opens with a chapter by Antonio Toral, Andreas Van
Cranenburgh and Tia Nutters which investigates ‘Literary Adapted Machine
Translation in a Well-Resourced Language Pair’. Building on earlier work
on the performance of relatively low-resource domain-adapted MT in lit-
erary translation, they train an English-to-Dutch NMT system on over
400 English novels and their Dutch translations, and 5,000 Dutch novels,
then compare its performance both to the generic DeepL system and to the
texts produced by professional translators, to assess whether the effort of
creating such extensive training resources is worthwhile. Next, in ‘“I Am a
Bit Surprised”: Literary Translation and Post-Editing: Cognitive Processes
Compared’, Waltraud Kolb uses keylogging and audio recording techniques
to compare the cognitive effort of five professional translators translating a
Hemingway short story into German, with five different professionals post-
editing an MT-derived version of the same text. She identifies differences
between the two modes in the type of engagement between the translators
and the source text, as well as impacts from the stimulus of the MT version
on the translator’s voice, along with potential changes to literary language
itself. Despite the obvious improvements we are seeing in MT capability,
it is worth reflecting on whether MT systems will ever be able to incorpo-
rate the type of world knowledge in their decision-making processes that
we see the human experts using in Kolb’s chapter. Finally, in ‘Mark My
Keywords: A Translator-Specific Exploration of Style in Literary Machine
Translation’, Dorothy Kenny and Marion Winters present a keyword
analysis of a novel post-edited by the internationally acclaimed translator
Hans-Christian Oeser. The analysis identifies words that are key in Oeser’s
post-edited text compared to the machine-translated version. It goes on to
investigate whether these keywords are characteristic of Oeser’s broader
translation work and of German literary fiction in general.
Part 2, containing six chapters on Machine Translation Applications
in Literary Translation, starts with a global assessment by James Hadley
of ‘MT and CAT: Challenges, Irrelevancies, or Opportunities for Literary
Translation?’, which investigates whether MT is likely to challenge literary
translators in the short to medium term, and looks towards the develop-
ment of CAT tools specifically designed to assist them. This is followed
by a case study, ‘Retranslating Proust Using CAT, MT and Other Tools’,
in which Andrew Rothwell describes his use of current translation tech-
nologies to retranslate into English Marcel Proust’s classic 1923 novel La
Prisonnière. The CAT editing environment of Déjà Vu X3 enables, for
each segment, a close comparison between the canonical translation pub-
lished in 1929 by C.K. Scott Moncrieff and a new interactive MT version
generated by DeepL. This, he argues, brings translation issues into focus
and helps identify creative solutions, taking away some of the spadework
of translating from scratch and allowing the translator to concentrate on
higher-level issues of style and literary effect. Antoni Oliver then presents
an experiment to develop ‘Author-Tailored Neural Machine Translation
Systems for Literary Works’, translated from English into Spanish and
Catalan, for the use of students and language learners. After a general
MT engine has been trained, it is tuned using a parallel corpus of human-
translated works by the same author, and the results are evaluated using
automated metrics, comparison with the output of free online NMT sys-
tems, and various measures of post-editing effort. The following chapter
by Shuyin Zhang shifts the focus of investigation from author to genre,
and from European languages to Chinese. ‘The Machine Translation of
Chinese Fantasy (Xianxia) Novels: An Investigation into the Leading
Websites Translating Net Literature from Chinese to English’ examines
the features of Xianxia novels that make MT particularly suitable, with
pre- and post-editing by their extensive fan-base, to support online inter-
cultural communication and the dissemination of Chinese literature. Next,
Introduction 17
the focus shifts again to MT in the translation of poetry, widely assumed,

because of features such as figurative language and rhyme, to be the sum-
mit of purely human linguistic endeavour. In ‘Up and About, or Betwixt
and Between? The Poetry of a Translation Machine’, Tim Van de Cruys
gives an introductory presentation of NMT technology, before going on
to describe a fully automatic system for poetry generation based on NNs,
and exploring how current MT models can be augmented with additional
constraints to enable the translation of literary texts. To conclude this part,
Aletta G. Dorst’s chapter ‘Metaphor in Literary Machine Translation:
Style, Creativity and Literariness’ brings together insights from Metaphor
Studies and Translation Studies to assess, through a systematic comparison
of English―Dutch MT and human translations of metaphor in literary
texts, whether MT leads to more erroneous or unsuitable metaphor trans-
lations or actually shows less interference in terms of deformation of sty-
listic patterns or normalisation of creativity. Central to her analysis is the
issue of how human and machine translation differs in the degree to which
they alter the metaphors’ stylistic coherence, creativity and literariness.
Part 3 is devoted to Corpus Linguistics, Text-Visualisation and Literary
Translation and explores the different benefits of using electronic corpora,
corpus linguistic tools and text-visualisation techniques to derive insights
into the style and linguistic structures of source texts. In the first of three
chapters, Angelika Peljak-Łapińska demonstrates the use of the KonText
corpus-analysis tool to investigate phraseology in the trilingual English-
Polish-Belarusian (EPB) Parallel Literary Corpus, showing how transla-
tors into less-resourced languages (in this case, Belarusian) can use it to
expand their phraseological competence. Next, Lisa Horenberg presents
a case study of the application of the Voyant Tools text-analysis suite to
the translation from English into Dutch of a short story by Evelyn Waugh.
Her chapter illustrates the enhanced appreciation of ST patterns that the
tools make available to the translator through a range of visualisations,
allowing translation decisions to be based on a deeper and more objective
understanding of the original. Finally, Tereza Šplíchalová uses corpus tech-
niques to explore literary style through the lens of fictional worlds theory,
which treats the imaginary ‘worlds’ generated by literary texts as seman-
tic constructs with extensional and intensional properties and structures.
She shows how analysing the individual narrative text as a micro-corpus
allows recovery of its salient intensional features for transfer into a trans-
lated version in a way that just reading it does not.
The fourth and final part of the book, focuses on Applying Specialised
Electronic Tools to Literary Translation. In Chapter 13, Avraham Roos
describes a Swansea workshop experiment inspired by the famously ludic
Le Ton Beau de Marot, published by Douglas R. Hofstadter in 1997. He
shows how participants with no knowledge of Hebrew could be guided by a
range of CALT tools, including corpora containing non-literary translations

of the text and online dictionaries, to construct rhyming verse translations
into English (also Polish and Chinese) of a short 11th-Century Hebrew
poem. In Chapter 14, the research team of Sasha Rudan, Eugenia Kelbert,
Sinisha Rudan, Lazar Kovacevic and Matthew Reynolds bring the book to
an optimistic conclusion by looking ahead to a new generation of CALT
tools deriving from a paradigm shift in their design. Their discussion draws
on examples from their own corpus analysis tools Bukvik and LitTerra,
used to compare multiple former Yugoslavian and Russian translations
of Charlotte Brontë’s Jane Eyre. They argue that the incorporation of
these and other analytical features into a future computerised translation
environment will finally ensure that the ‘A’ in CALT comes to stand for
‘Augmented.’
Acknowledgements
The ADAPT Centre for Digital Content Technology (www.adaptcentre
.ie) at Dublin City University is funded by the Science Foundation Ireland
Research Centres Programme (Grant 13/RC/2106) and is co-funded by the
European Regional Development Fund.
References
Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly
learning to align and translate. In 3rd International Conference on Learning
Representations, ICLR 2015, Conference Track Proceedings, San Diego (15p).
https://doi.org/10.48550/arXiv.1409.0473
Baker, M. (1993). Corpus linguistics and translation studies: Implications and
applications. In M. Baker, G. Francis, & E. Tognini-Bonelli (Eds.), Text and
technology: In honour of John Sinclair (pp. 233–50). John Benjamins. https://
doi.org/10.1075/z.64.15bak
Bentivogli, L., Bisazza, A., Cettolo, M., & Federico, M. (2016). Neural versus
phrase-based machine translation quality: A case study. In Proceedings of the
2016 Conference on Empirical Methods in Natural Language Processing,
Austin (pp. 257–67). https://aclanthology.org/D16-1025
Besacier, L. (2014). Traduction automatisée d’une œuvre littéraire: Une étude
pilote. Traitement Automatique du Langage Naturel (TALN). Marseille.
https://hal.inria.fr/hal-01003944
Boase-Beier, J. (2010). Stylistic approaches to translation. Routledge. https://doi
.org/10.4324/9781315759456.
Brokaw, C., & Kornicki, P. (2016). The history of the book in East Asia. Routledge.
https://doi.org/10.4324/9781315239200
Castilho, S., Moorkens, J., Gaspari, F., Calixto, I., Tinsley, J., & Way, A. (2017).
Is neural machine translation the new state-of-the-art? Prague Bulletin of
Mathematical Linguistics, 108, 109–120. https://core.ac.uk/download/pdf
/195384513.pdf
Introduction 19
Cheesman, T., Flanagan, K., Thiel, S., Rybicki, J., Laramee, R.S., Hope, J., &
Roos, A. (2017). Multi-retranslation corpora: Visibility, variation, value, and
virtue. Digital Scholarship in the Humanities, 32(4), 739–60. https://www
.academia.edu/28423745/Multi_Retranslation_Corpora_Visibility_Variation
_Value_and_Virtue
Cho, K., Van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk,
H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–
decoder for statistical machine translation. In Proceedings of the 2014 Conference
on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar
(pp. 1724–34). Association for Computational Linguistics. https://aclanthology
.org/D14-1179/
Cronin, M. (2010). The translation crowd. Tradumàtica: traducció i tecnologies
de la informació i la comunicació, 8. https://revistes.uab.cat/tradumatica/article
/view/n8-cronin/pdf_15
Daems, J. (2022). Dutch literary translators’ use and perceived usefulness of
technology. In J. Hadley, K. Taivalkoski-Shilov, C. Teixeira, & A. Toral (Eds.),
Using technologies for creative-text translation (pp. 40–65). Routledge. https://
doi.org/10.4324/9781003094159
Eagleton, T. (2014). How to read literature. Yale University Press. https://tinyurl
.com/4x5xxr2j
Fabb, N. (2010). Is literary language a development of ordinary language? Lingua,
120(5), 1219–32. https://doi.org/10.1016/j.lingua.2009.07.007
Farrell, M. (2018). Machine translation markers in post-edited machine translation
output. In Proceedings of the 40th Conference Translating and the Computer,
London (pp. 50–59). https://www.asling.org/tc40/wp-content/uploads/TC40
-Proceedings
Forcada, M. (2017), Making Sense of neural machine translation. Translation
Spaces, 6(2), 291–309. https://doi.org/10.1075/ts.6.2.06for
Forcada, M., & Ñeco, R. (1997). Recursive hetero-associative memories for
translation. In Biological and Artificial Computation: From Neuroscience to
Technology (International Work-Conference on Artificial and Natural Neural
Networks, IWANN’97, Proceedings), Lanzarote, Canary Islands, Spain (pp.
453–62). https://link.springer.com/chapter/10.1007/BFb0032504
Genzel, D., Uszkoreit, J., & Och, F.J. (2010). ‘Poetic’ statistical machine
translation: Rhyme and meter. In Proceedings of the 2010 Conference on
Empirical Methods in Natural Language Processing, Cambridge, MA (pp.
158–66). https://aclanthology.org/D10-1016/
Greene, E., Bodrumlu, T., & Knight, K. (2010). Automatic analysis of rhythmic
poetry with applications to generation and translation. In Proceedings of the
Cambridge, MA (pp. 524–33). https://aclanthology.org/D10-1051/
Guerberof-Arenas, A., & Toral, A. (2019). Measuring readers’ engagement
in literary texts: A study comparing human translation to machine assisted
translation. In Proceedings of the 2019 Congress of the European Society for
Translation Studies, Stellenbosch, South Africa.
Hadley, J.L., Taivalkoski-Shilov, K., Teixeira, C.S.C., & Toral, A. (Eds.). (2022).
Using technologies for creative-text translation. Routledge. https://doi.org/10
.4324/9781003094159
Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J., Federmann, C., Huang,
X., Junczys-Dowmunt, M., Lewis, W., Li, M., Liu, S., Liu, T-Y., Luo, R.,
Menezes, A., Qin, T., Seide, F., Tan, X., Tian, F., Wu, L., … Zhou, M. (2018).
Achieving human parity on automatic Chinese to English news translation.
ArXiv. https://arxiv.org/abs/1803.05567
Khayrallah, H., & Koehn, P. (2018). On the impact of various types of noise on
neural machine translation. In Proceedings of the 2nd Workshop on Neural
Machine Translation and Generation, Melbourne, Australia (pp. 74–83).
Association for Computational Linguistics. https://aclanthology.org/W18-2709/
Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In
MT Summit X, Conference Proceedings: The Tenth Machine Translation Summit,
Phuket, Thailand (pp. 79–86). https://aclanthology.org/2005.mtsummit-papers.11/
Koehn, P. (2020). Neural machine translation. Cambridge University Press. https://
doi.org/10.1017/9781108608480
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N.,
Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin,
A., & Herbst, E. (2007). Moses: Open source toolkit for statistical machine
translation. In Proceedings of the 45th Annual Meeting of the Association for
Computational Linguistics Companion Volume Proceedings of the Demo and
Poster Sessions, Prague, Czech Republic (pp. 177–80). https://aclanthology.org
/P07-2045/
Koehn, P., Och, F.J., & Marcu, D. (2003). Statistical phrase-based translation.
In Proceedings of the 2003 Human Language Technology Conference of the
North American Chapter of the Association for Computational Linguistics,
Edmonton, Alberta, Canada (pp. 127–33). https://aclanthology.org/N03-1017/
Kuzman, T., Vintar, Š., & Arčan, M. (2019). Neural machine translation of literary
texts from English to Slovene. In Proceedings of Machine Translation Summit
XVII: The Qualities of Literary Machine Translation, Dublin, Ireland (pp. 1–9).
European Association for Machine Translation. https://aclanthology.org/W19
-7301.pdf
Landers, C. (2001). Literary translation: A practical guide. Multilingual Matters.
https://www.perlego.com/book/571617/literary-translation-a-practical-guide
-pdf
Lee, T. (2011). The death of the translator in machine translation. Target:
International Journal of Translation Studies, 23(1), 92–112. https://doi.org/10
.1075/target.23.1.06lee
Littau, K. (2011). First steps towards a media history of translation. Translation
Studies, 4(3), 261–81. https://www.tandfonline.com/doi/abs/10.1080
/14781700.2011.589651
Lommel, A. (2018). Augmented translation: A new approach to combining
human and machine capabilities. In Proceedings of the 13th Conference of the
Association for Machine Translation in the Americas (Volume 2: User Track),
Boston, MA (pp. 5–12). https://aclanthology.org/W18-1905
Matusov, E. (2019). The challenges of using neural machine translation for
literature. In Proceedings of Machine Translation Summit XVII: The Qualities
of Literary Machine Translation, Dublin, Ireland (pp. 10–19). European
Association for Machine Translation. https://aclanthology.org/W19-7302
Introduction 21
Meldrum, Y. (2009). Translationese in Japanese literary translation. TTR:

Traduction, Terminologie, Redaction, 22(1), 93–118. https://doi.org/10.7202
/044783ar
Moniz, H., & Parra Escartín, C. (Eds.). (2023). Towards responsible machine
translation: Ethical and legal considerations in machine translation. Springer.
Moorkens, J., Toral, A., Castilho, S., & Way, A. (2018). Translators’ perceptions
of literary post-editing using statistical and neural machine translation.
Translation Spaces, 7(2), 240–62. https://doi.org/10.1075/ts.18014.moo
Moretti, F. (2013). Distant Reading. Verso. https://www.versobooks.com/books
/1421-distant-reading
Newmark, P. (2004). Non-literary in the light of literary translation. The Journal
of Specialised Translation, 1, 8–13. https://www.jostrans.org/issue01/art_
newmark.pdf
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang,
C.X., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F.,
Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., &
Lowe, R. (2022). Training language models to follow instructions with human
feedback. arXiv. https://arxiv.org/abs/2203.02155
O’Hagan, M. (Ed.). (2020). The Routledge handbook of translation and technology.
Routledge. https://doi.org/10.4324/9781315311258
Papineni, K., Roukos, S., Ward, T., & Zhu, W-J. (2002). BLEU: A method for
automatic evaluation of machine translation. In ACL–2002: 40th Annual
meeting of the Association for Computational Linguistics, Philadelphia, PA,
Association for Computational Linguistics (pp. 311–18). https://aclanthology
.org/P02-1040/
Poncelas, A., Shterionov, D., Way, A., Maillette de Buy Wenniger, G., & Passban,
P. (2018). Investigating backtranslation in neural machine translation. In
Proceedings of the 21st Annual Conference of the European Association for
Machine Translation (EAMT 2018), Alicante, Spain (pp. 249–58). https://doi
.org/10.48550/arXiv.1804.0618
Popović, M. (2015). chrF: Character n–gram F–score for automatic MT evaluation.
In Proceedings of the Tenth Workshop on Statistical Machine Translation,
Lisbon, Portugal (pp. 392–95). https://aclanthology.org/W15-3049.pdf
Rothwell, A., & Youdale, R. (2022) Computer-Assisted Translation (CAT) tools,
translation memory and literary translation. In S. Deane-Cox & A. Spiessens
(Eds.), The Routledge handbook of translation and memory (pp. 381–402).
Ruffo, P. (2022). Collecting literary translators’ narratives: Towards a new
paradigm for technological innovation in literary translation. In J.L. Hadley,
K. Taivalkoski-Shilov, C.S.C. Teixeira, & A. Toral (Eds.), Using technologies
for creative-text translation (pp. 18–39). Routledge. https://doi.org/10.4324
/9781003094159
Ruiz Zafón, C. (2011). El Prisionero del Cielo. Planeta. https://www.planetadelibros
.com/libro-el-prisionero-del-cielo/221324
Sennrich, R., Haddow, B., & Birch, A. (2016). Improving neural machine
translation models with monolingual data. In Proceedings of the 54th Annual
Meeting of the Association for Computational Linguistics (Volume 1: Long
Papers), Berlin, Germany (pp. 86–96). https://aclanthology.org/P16-1009/
Slessor, S. (2020). Tenacious technophobes or nascent technophiles? A survey of

the technological practices and needs of literary translators. Perspectives, 28(2),
238–52. https://doi.org/10.1080/0907676X.2019.1645189
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., & Makhoul, J. (2006). A
study of translation edit rate with targeted human annotation. In Proceedings
of the 7th Conference of the Association for Machine Translation in the
Americas: Technical Papers, Cambridge, MA (pp. 223–31). Association for
Machine Translation in the Americas. https://aclanthology.org/2006.amta
-papers.25/
Tezcan, A., Daems, J., & Macken, L. (2019). When a ‘sport’ is a person and other
issues for NMT of novels. In Proceedings of Machine Translation Summit XVII:
The Qualities of Literary Machine Translation, Dublin, Ireland (pp. 40–49).
https://aclanthology.org/W19-7306/
Toral, A. (2019). Post-editese: An exacerbated translationese. In Proceedings of
MT Summit XVII, Volume 1: Research Track, Dublin, Ireland (pp. 273–81).
-6627/
Toral, A., Castilho, S., Hu, K., & Way, A. (2018). Attaining the unattainable?
Reassessing claims of human parity in neural machine translation. In Proceedings
of the Third Conference on Machine Translation: Research Papers, Brussels,
Belgium (pp. 113–23). https://aclanthology.org/W18-6312/
Toral, A., Oliver, A., & Ribas Ballestín, P. (2020). Machine translation of novels
in the age of transformer. In J. Porsiel (Ed.), Maschinelle Übersetzung für
Übersetzungsprofis (pp. 276–95). BDÜ Fachverlag. https://doi.org/10.48550/
arXiv.2011.14979
Toral, A., & Way, A. (2015a). Translating literary text between related languages
using SMT. In Proceedings of the Fourth Workshop on Computational
Linguistics for Literature, NAACL, Denver, CO (pp. 123–32). https://
aclanthology.org/W15-0714.pdf
Toral, A., & Way, A. (2015b). Machine-assisted translation of literary text: A case
study. Translation Spaces, 4(2), 240–67. https://doi.org/10.1075/ts.4.2.04tor
Toral, A., & Way, A. (2018). What level of quality can neural machine translation
attain on literary text? In J. Moorkens, S. Castilho, F. Gaspari, & S. Doherty
(Eds.), Translation quality assessment: From principles to practice (pp. 263–
87). Springer. https://arxiv.org/abs/1801.04962
Toral, A., Wieling, M., & Way, A. (2018). Post-editing effort of a novel with
statistical and neural machine translation. Frontiers in Digital Humanities, 5
(Art. 9), 1–11. https://doi.org/10.3389/fdigh.2018.00009
Toury, G. (2012). Descriptive translation studies—And beyond. John Benjamins.
https://doi.org/10.1075/btl.4
Trancoso, I. (2022). Treating speech as personable identifiable information—
Its impact in machine translation. In H. Moniz & C. Parra Escartín (Eds.),
Towards responsible machine translation: Ethical and legal considerations in
machine translation (pp. 215–33). Springer. https://link.springer.com/chapter
/10.1007/978-3-031-14689-3_1
Van der Meer, J. (2021). Translation economics of the 2020s: A journey into the
future of the translation industry in eight episodes. Multilingual Magazine,
Introduction 23
July/Aug 2021. https://multilingual.com/issues/july-august-2021/translation

-economics-of-the-2020s/
Vanmassenhove, E, Shterionov, D., & Gwilliam, M. (2021). Machine translationese:
Effects of algorithmic bias on linguistic complexity in machine translation. In
Proceedings of the 16th Conference of the European Chapter of the Association
for Computational Linguistics: Main Volume, online (pp. 2203–13). https://
aclanthology.org/2021.eacl-main.188/
Vanmassenhove, E., Shterionov, D., & Way, A. (2019). Lost in translation: Loss
and decay of linguistic richness in machine translation. In Proceedings of
MT Summit XVII, Volume 1: Research Track, Dublin, Ireland (pp. 222–32).
-6622.pdf
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, Ll, Gomez, A.N., Kaiser,
L., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural
Information Processing Systems 30: Annual Conference on Neural Information
Processing Systems 2017, Long Beach, CA (pp. 5998–6008). https://doi.org/10
.48550/arXiv.1706.03762
Voigt, R., & Jurafsky, D. (2012). Towards a Literary Machine Translation: The Role
of Referential Cohesion. In Proceedings of the NAACL-HLT 2012 Workshop
on Computational Linguistics for Literature, pages 18–25, Montréal, Canada.
Association for Computational Linguistics.
Way, A. (2018). Quality expectations of machine translation. In J. Moorkens, S.
Castilho, F. Gaspari, & S. Doherty (Eds.), Translation quality assessment: From
principles to practice (pp. 159–78). Springer. https://doi.org/10.48550/arXiv
.1803.08409
Way, A. (2020). Machine translation: Where are we at today? In E. Angelone,
G. Massey, & M. Ehrensberger-Dow (Eds.), The Bloomsbury companion
to language industry studies (pp. 311–32). Bloomsbury. https://tinyurl.com
/2bjxtb9r
Way, A. (forthcoming, 2024). What does the future hold for translation technologies
in society? In S. Baumgarten and M. Tieber (Eds.), Routledge handbook of
translation technology and society. Routledge.
Way, A., Rothwell, A., & Youdale, R. (2023). Why literary translators should
embrace translation technology. Tradumática. (In press).
Webster, R., Fonteyne, M., Tezcan, A., Macken, L., & Daems, J. (2020). Gutenberg
goes neural: Comparing features of Dutch human translations with raw neural
machine translation outputs in a corpus of English literary classics. Informatics,
7(3), 21p. https://doi.org/10.3390/informatics7030032
Youdale, R. (2020). Using computers in the translation of literary style: Challenges
and opportunities. Routledge. https://doi.org/10.4324/9780429030345
Yum, H. (2011). History and function of dispersion aids used in traditional East
Asia papermaking. Journal of the Institute of Conservation, 34(2), 202–208.
https://doi.org/10.1080/19455224.2011.607008
Part 1
The Automated and Post-

Edited Machine Translation of
Literature

1 Literary-Adapted Machine Translation
in a Well-Resourced Language Pair
Explorations with More Data and Wider
Contexts
Antonio Toral, Andreas van Cranenburgh,

and Tia Nutters
1.1 Introduction
Research on machine translation (MT) for literary texts has recently gained
a foothold in the field. A popular strand therein leverages well-established
domain adaptation techniques to tailor neural MT (NMT) systems to the
literary domain (Toral and Way, 2018; Kuzman et al., 2019, Matusov,
2019). To the best of our knowledge, the largest amount of literary in-
domain data used to date is 1.1 million bilingual sentence pairs (Toral
and Way, 2018) and 10 million monolingual sentences (Matusov, 2019).
Our study tackles a relatively well-resourced language pair and goes a step
beyond this previous work in terms of the amount of data used: we train
an English-to-Dutch MT system on around 5 million sentence pairs and
over 20 million monolingual target-language sentences.
In addition, we note that the MT systems trained in the aforementioned
studies deal with sentences in isolation, i.e. they translate one sentence at
a time, independently of the surrounding sentences. However, it has been
shown that literary texts have particularly dense reference chains (Voigt and
Jurafsky, 2012), so incorporating discourse features beyond the level of the
sentence is an important research focus for applying MT to literary texts.
Accordingly, in this study, we also build MT systems that go beyond the sen-
tence level. These are usually referred to in the literature as document-based
MT systems (Popel et al., 2019) and, to the best of our knowledge, have not
yet been explored for literary texts except for Baas (2021).
The word literary, such as in the established term ‘literary MT’, is often
used to refer to fiction novels in general. However, in this chapter, we are
also interested in a narrower, more exclusive sense of the word when used
in the phrase ‘literary fiction’ which we elaborate on here. In the rest of this
chapter, we use the term novel for fiction novels in general which also form
the domain we consider. In the book market, publishers attribute a genre
label to books as a means to communicate with their prospective readers.
Usually, a distinction is being made between genre fiction, such as romance
DOI: 10.4324/9781003357391-3
28 Antonio Toral et al.
or suspense, and general fiction, also often called literary fiction (Squires,
2009). Seen from this perspective, literary fiction is anything that is not
clearly belonging to a certain genre (cf. Squires, 2009, pp. 4–5; Baldick,
2015). What is literary is difficult to define; in the words of Squires (2009,
p. 5): ‘Literary fiction is that published by literary imprints such as [she
names several publishing houses here]’. This circular way of reasoning has
been quite common in book history and literary studies. Other theories
claim that specific formal features determine a text’s literariness (Russian
Formalism; cf. Baldick, 2008; see also Section I.1, Introduction, this vol-
ume for more on the literary/non-literary distinction). In contrast, the pro-
ject The Riddle of Literary Quality1 took an empirical and computational
approach. The concept of literature was triangulated in a data-driven man-
ner using a corpus of novels, a reader survey, and machine learning. The
large reader survey asked respondents how literary they found the novels
(Koolen et al., 2020), and machine learning was applied to estimate to
what extent the text of the novels can explain the reader’s judgments (Van
Cranenburgh et al., 2019). In our chapter, we apply this machine learning
approach to explore the literariness of translations (see Section 1.4.1 for
more details).
In this work we aim to answer four main research questions:
RQ1. Is it worthwhile building a literary-adapted MT system for a well-

resourced language pair, for which generic systems (e.g. DeepL) are
known to be competitive? Evidence shows that such an in-domain
system performs much better than an online general-purpose
system for a language pair with scarce resources, e.g. English-to-
Catalan (Toral et al., 2020). We aim to find out whether this is
also the case for a better-resourced language pair.
RQ2. How do sentence- and document-based MT systems compare?
Given the findings by Voigt and Jurafsky (2012), we expect doc-
ument-level literary-adapted MT to perform better than the sen-
tence-level variety.
RQ3. How literary and varied are the translations of the MT systems
compared to those produced by professional literary translators?
We deem this relevant since a literary translation should not only
be accurate and fluent but should also preserve the literary charac-
ter of the original.
RQ4. To what extent is literariness prediction useful for analysing MT
output? Are the literariness predictions consistent with human
judgments?
1 https://www.huygens.knaw.nl/en/projecten/the-riddle-of-literary-quality-2/.
Literary-Adapted Machine Translation 29
1.2 Methodology and Experimental Setup

1.2.1 Datasets
The datasets used to train, develop, and evaluate the MT systems are listed
in Table 1.1. The training data includes both in-domain corpora (parallel
English–Dutch and monolingual Dutch novels) and general-domain cor-
pora (the first 20 million sentence pairs from Paracrawl v9,2 a parallel
corpus automatically compiled from web text). The in-domain corpora
consist mostly of popular novels; although they include literary novels,
genre fiction (suspense, sci-fi, fantasy, romance) predominates. The devel-
opment set (see Section 1.2.4) consists of one English-language novel
(Boyne, 2006) and its published Dutch translation (Boyne, 2010), chosen
as a representative example of recent popular fiction. Finally, the test set
(see Section 1.2.3) consists of 31 book pairs, chosen to be a representative
sample of fiction, as it encompasses different periods (from 1884 to 2012,
M = 1983, Mdn = 2007) as well as different varieties (suspense, romance,
and literary novels). The list of novels in the test set is shown in Appendix
A (see Section I.2, Introduction, this volume, for more on how data is split
into training, development, and test sets).
Table 1.1 Dataset sizes (words, on the source and target sides).
Dataset # # Sentences or # Words

Novels sentence pairs
English Dutch
Training: parallel 500 4,963,665 60,931,780 65,050,015
books
Training: 4,435 22,659,100 *261,332,497 276,748,103
monolingual
books
Training: paracrawl ― 20,000,000 290,659,203 290,533,746
Development 1 2,919 47,196 47,566
Test 31 305,989 3,989,141 4,263,532
*means that the English side of this corpus is synthetic, created by means of back-translation
(see Section 1.2.3).
2 This dataset (https://paracrawl.eu/) contains around 89 million sentence pairs. A subset of

20 million was used for two reasons: (i) early experimentation using a bigger subset did
not lead to improvements, and (ii) using the whole set would substantially increase train-
ing time.
1.2.2 Pre- and Post-Processing
We separate words from adjacent punctuation symbols with scripts from

Moses (Koehn et al., 2007; English) and Alpino (van Noord, 2006; Dutch).
We normalise punctuation variants to one form with Moses; e.g. ‘,’, «
and » ‘become’. The parallel novels are sentence-aligned with Vecalign
(Thompson and Koehn, 2019), which mainly results in 1-to-1 pairs
(89.8%), but also some many-to-1 and 1-to-many. Using Vecalign, we
keep sentence pairs with cosine distance lower than 0.7. This means that
each sentence is represented as a list of numbers (i.e. a vector) and for each
sentence pair, we measure how different the lists that represent the English
and Dutch sides are and keep the pair if it is lower than the threshold 0.7,
which leads to just 2.3% of the pairs being discarded. Prior to training and
translating, the data is segmented into subword units (48,000 operations
jointly on both the source and target languages) with SentencePiece (Kudo
and Richardson, 2018; see Section I.2, Introduction, this volume, for more
on why NMT system building needs to be sped up).
1.2.3 MT Evaluation Metrics
We use two automatic metrics to estimate the translation quality of the

systems used in our experiments. BLEU (Papineni et al., 2002) is the most
widely used evaluation metric in MT and is based on lexical overlap
between the MT output and a ‘gold-standard’ human translation. COMET
(Rei et al., 2020) is a newer metric that is gaining popularity and correlates
with human judgments better than BLEU (Kocmi et al., 2021). COMET is
trained to approximate human judgments by learning from examples using
a multilingual language model (see Section I.2, Introduction, this volume,
for more on measuring translation quality).
COMET also offers reference-free quality estimation (COMET QE;
Rei et al., 2020); i.e. a variant of the COMET automatic MT metric that
scores an MT output with respect to the original without having access to
a human translation; clearly, this is the normal situation one is confronted
with in actual translation use-cases. We use COMET QE to select the best
MT output when there are multiple candidates.
1.2.4 MT System Building
We train several Transformer models (Vaswani et al., 2017) using dif-

ferent subsets of the training data (see Section 1.2.1) with two different
network sizes (Transformer base and Transformer big). Transformer is the
state-of-the-art neural network for NMT at the time of writing. The neural
networks in their original base and big variants contain 65M and 213M
trainable weights (or ‘parameters’), respectively. In addition, we train sys-
tems with two different instance granularities: systems are either trained
on isolated sentences (one at a time) or on longer blocks made up of a few

consecutive sentences.
All our systems are trained with Marian3 v1.9.0 (Junczys-Dowmunt
et al., 2018)—one of the leading NMT toolkits—on a single Nvidia V100
GPU. All systems are trained with early stopping: at fixed intervals during
training, the model is evaluated on the development set, if for five con-
secutive times the BLEU score obtained is no better than the highest one
attained to date, the training procedure is stopped. For each system we
build an ensemble (‘Combo’, below) of the two models that perform best
on the development set according to BLEU.
In the following points, we outline each of the MT systems built:
• S1. Trained on the in-domain parallel data using the Transformer base
network.
• S2. Compared to S1, adds the in-domain monolingual data previ-
ously back-translated (Sennrich et al., 2016; Poncelas et al., 2018; see
Section I.2, Introduction, this volume, for more on synthetic data crea-
tion) using a reverse version (i.e. Dutch-to-English) of S1. After initial
training finishes, the system is fine-tuned on the in-domain parallel data.
Fine-tuning refers to a second training phase. Usually, the first train-
ing phase involves larger amounts of general-domain data, while the
fine-tuning phase uses smaller amounts of domain-specific data. In our
case, the first phase uses all training data while the second uses only the
parallel novels. The remaining systems are fine-tuned in the same way.
• S2Big. Equal to S2 except for using Transformer big.
• S3Big. Adds to S2Big the general-domain parallel data.
• S3BigDocX00. Equal to S3Big except for the fact that fine-tuning is per-
formed at document level, up to either 400 or 800 characters on each
side, following Baas (2021).
• Combo. An ensemble of S3Big, S3BigDoc400 and S3BigDoc800.
The motivation for building ensembles like Combo is that its underlying
systems may have complementary strengths since they were trained with
different granularities (isolated sentences, and small and larger amounts
of multiple consecutive sentences). For each sentence, Combo selects the
translation from the three MT systems with the highest quality according
to COMET QE.
We consider three baselines in our study, namely, two popular gen-
eral-purpose online systems (DeepL4 and Google Translate5), as well as
3 https://marian-nmt.github.io/.
4 https://www.deepl.com/translator.
5 https://translate.google.com/.
a state-of-the-art open-source system trained on publicly available data,

Opus-MT6 (Tiedemann & Thottingal, 2020).
1.3 Automatic Evaluation
Table 1.2 shows the results for BLEU and COMET, as well as the training
time taken by each system. We observe that the use of additional training
data, be it monolingual in-domain (S2 vs S1) or parallel general-purpose
(S3Big vs S2Big), results in consistent improvements according to both
BLEU and COMET. The same trend can be observed when increasing the
size of the network (S2Big vs S2). We also trained a Transformer big vari-
ant of S1. However, this did not lead to an improvement, suggesting that
the additional network parameters in the S2Big variant offer an edge only
when sufficient amounts of training data are available.
However, the two metrics did behave differently when comparing a sys-
tem trained on isolated sentences to a variant trained on multiple consecu-
tive sentences (S3Big vs S3BigDoc400 and S3BigDoc800): BLEU slightly
prefers the former while COMET prefers the latter. Given that COMET
correlates better with human judgments as well as the positive outcomes
of document-level MT for the translation of pronouns (e.g. Baas, 2021),
we decided to give COMET the benefit of the doubt in this case. Finally,
a combination based on system selection (taking as inputs the MT out-
puts produced by S3Big, S3BigDoc400 and S3BigDoc800) leads to the best
Table 1.2 Results with automatic evaluation metrics (BLEU and COMET estimate
translation quality; higher is better).
MT system BLEU ▵% BLEU COMET ▵% Training time

COMET (h)
DeepL 33.83 ― 0.512 ― ―

Opus-MT 27.43 –23.32% 0.346 –48.08% ―
S1 32.63 –3.67% 0.461 –10.89% 24 + 0
S2 33.52 –0.92% 0.473 –8.14% 58 + 9
S2Big 34.41 1.71% 0.488 –4.83% 174 + 16
S3Big 35.63 5.07% 0.500 –2.32% 598 + 20
S3BigDoc400 35.29 4.14% 0.505 –1.31% 598 + 20
S3BigDoc800 35.52 4.78% 0.504 –1.57% 598 + 24
Combo 35.46 4.61% 0.533 3.93% 598 + 64
▵% denotes relative differences with respect to DeepL. MT training times given as hours for
first training plus hours for fine-tuning. Best result per metric shown in bold.
6 https://github.com/Helsinki-NLP/Opus-MT.
result in terms of COMET, but it does not lead to an improvement in

BLEU (see final row ‘Combo’).
Our best system in terms of COMET (Combo) results in a 4% relative
improvement over the competitive online system DeepL overall. The other
online system considered, Google Translate, was tested on the two shortest
books of the test set. In both cases, its COMET scores were considerably
lower than DeepL’s (0.5134 vs 0.5362 for Golding’s and 0.495 vs 0.5121
for Hemingway’s). Due to this, and because we estimated the cost of trans-
lating all books with Google Translate to be rather high (around €400,
compared to €20 for DeepL), we decided not to translate the remaining
books with Google Translate.
Next, we zoom in and present the COMET scores per MT system and
book (Figure 1.1). Our Combo system obtains higher COMET scores than
DeepL on 25 out of the 31 books in the test set. For three of those books,
our system improves by over 10%: Rowling (30.4%), French, (14%) and
Tolkien (12%). The six books for which DeepL outperforms our best sys-
tem are those by Auster (–2%), James (–2%), Joyce (–6%), Roth (–1%),
Steinbeck (–6%) and Twain (–19%). In the last book, the large difference
in performance may be due to our system not being exposed to late 19th-
century English (Twain, dated 1884, is the oldest book in our test set). In
terms of the other baseline, Opus-MT, it makes the most sense to compare
it to S1 and S2, since Opus-MT also uses the Transformer base network.
Even in this fairer comparison, Opus-MT still underperforms obtaining
the lowest scores on all books. Both automatic metrics notably favour our
systems, e.g. S1’s COMET score is 32% higher than that of Opus-MT. The
difference in performance between Opus-MT and S1/S2 might be attrib-
uted to using generic publicly available corpora versus parallel novels as
parallel training data.
1.4 Quantitative Analyses
1.4.1 Literariness Predictions
Van Cranenburgh et al. (2019) presented a machine-learning model that

predicts the degree of literariness of a text fragment. The model was
trained using judgments on a seven-point Likert scale (1 = least literary,
7 = most literary) from a large reader survey (Koolen et al., 2020) on
401 contemporary Dutch-language novels (both translated and original).
To avoid bias, respondents were not instructed on any definition of ‘lit-
erary’. Nonetheless, a substantial degree of agreement was observed; in
this way, Koolen et al. (2020) wanted to find out the readers’ own ideas
about literary quality, assuming that most readers do have an idea, and
specifically, that readers assess literary quality based on literary conven-
tions shared among authors and readers. Rather than defining criteria for
Figure 1.1 C
OMET scores by DeepL, Opus-MT and our Combo system on the 31
test set novels.
literariness, we take the survey ratings as a proxy and operationalise liter-

ariness with this predictive model. The model predicts the literary rating
for a novel from its textual features. These features consist of paragraph
vectors (Le and Mikolov, 2014)—a type of distributional semantic feature
that provides a mathematical summary of the meaning of short pieces of
text based on word co-occurrence—of 1000-word fragments, automati-
cally extracted from the text. Van Cranenburgh et al. (2019) showed that
the model is quite successful at predicting the perceived literariness of nov-
els. The predictions of the Paragraph Vector Model we use explain 42.9%
of the variation in ratings (R2; Field et al., 2012, p. 262). For a given novel,
the predicted rating is, on average, 0.74 points too low or high on the 1–7
Likert scale.
However, this evaluation was based on predicting the literary ratings
of different novels. In the present work, we are concerned with differ-
ent translations of the same work, which is arguably a more challeng-
ing task since the differences may be more subtle. Our aim is to explore
the predicted degrees of literariness across the translations and determine
whether they agree with the judgments from the human evaluation survey
(Section 1.5.2). Since the model expects 1000-word chunks, for each novel
we obtain a sequence of literariness predictions, which we reduce to a sin-
gle score per novel using the mean.
Given that DeepL is a general-purpose system, while our MT systems
have been fine-tuned on novels, we expected our systems to produce trans-
lations with higher literariness than DeepL. However, we find that DeepL
scores higher than our Combo system for 27 out of the 31 novels that
make up the test set; the four novels on which our Combo system scores
higher are Boyne, French, Tolkien, and Twain. The differences are small
but significant (p < 0.01, paired t-test; Field et al., 2012, pp. 386–93) for
26 out of the 31 novels, with Cohen’s d (Cohen, 1988, pp. 20–27) ranging
between 0.24 and 0.77, indicating small and medium effect sizes. For 17
out of 31 novels, the human translation (HT) is predicted to be more liter-
ary than any of the MT outputs (and these 17 novels have high literariness,
4.5 or higher) while the 3 least literary HTs are less literary than the MT
systems, indicating a wider range for HT (see Figure 1.2). The difference
between our sentence- and document-level systems is negligible. Lastly,
when comparing our Combo system to Opus-MT, translations by Combo
are predicted to be more literary for half the books, including the most
literary novels. The differences between the two systems are significant for
20 out of 31 novels (p < 0.01, paired t-test).
1.4.2 Lexical Variety
We use type-token ratio (TTR) to measure lexical variety in the trans-

lations. TTR is calculated by dividing the types (total number of differ-
ent words) occurring in a text by its tokens (total number of words). The
higher the ratio, the larger the vocabulary richness of the text, i.e. it can
be considered to be lexically more varied. Since TTR is sensitive to text
length, for each book we take a random sample of similar size in words to
the smallest book (Hemingway).
The sorted mean TTR scores are 0.1645 for HT, 0.1595 for DeepL,
0.1573 for Combo, and 0.1562 for Opus-MT. According to pairwise
Wilcoxon tests (Field et al., 2012, pp. 667–73)—similar to the paired
Student's t-test but more robust to outliers—all differences between adja-
cent translations are significant (all p < 0.001, except Combo―Opus-MT
Figure 1.2 M
ean of predicted literariness scores across fragments (error bars show
standard error; Fields et al., 2012: 42) for the human translation and
three MT systems.
p < 0.05). Overall, these results are consistent with those on literariness
(Section 1.4.1).
1.4.3 Correlations
Finally, we study whether there are any correlations between the auto-
matic evaluation metric COMET (Section 1.3) and the quantitative metrics
used for analyses, literariness (Section 1.4.1), TTR (Section 1.4.2) as well
as n-gram overlap. The latter was already explored by Toral and Way
(2018) to measure the novelty of a text with respect to the training data
(considered as representative of fiction), by checking how many of the
unique 4-grams (i.e. sequences of 4 consecutive words) in a novel are also
present in the training data. Thus, the lower the overlap, the more distinct
the novel is compared to the training data.
We find moderate Spearman correlations (Field et al., 2012, pp. 219–22)
for the difference between the COMET scores of the Combo system and
DeepL on the one hand, and n-gram overlap (ρ = 0.44, p < 0.05, Figure 1.3)
as well as the literariness of the human translation (ρ = –0.5, p < 0.01,
Figure 1.4) on the other. A crucial part of Figures 1.3 and 1.4 is the 0 point
on the x-axis; to the right of the shaded areas we find all the books on
Figure 1.3 L
inear correlation of the difference between the COMET scores of the
Combo system and DeepL, and n-gram overlap between our training
and test set.
Figure 1.4 L
inear correlation between the absolute difference between the
COMET scores of the Combo system and DeepL and predicted
literariness of the human translations.
which Combo scores better than DeepL. This means that on most books
with high literariness, our system performed better in terms of (i) a higher
level of n-gram overlap indicating an advantage of our Combo system
compared to DeepL translations, but (ii) higher literariness is associated
with less of an advantage for our Combo system compared to DeepL, i.e.
the correlation shows that the difference is smaller for books which are
deemed to be more literary. Note that Pearson (parametric) correlations
did not meet all the assumptions, which is why we report Spearman (non-
parametric). However, we still show linear regressions in Figures 1.3–1.5
as that makes the interpretation easier since they have the original values
in the two axes. The bands in the plots show the 95% confidence intervals
(Field et al., 2012, pp. 138, 211), to visualise how well the straight line
from the linear regression describes the observed data.
We find similar correlations between DeepL and Opus-MT, both in
the case of n-gram overlap (ρ = 0.44, p < 0.05; note that this refers to
n-gram overlap with the training data of our system, not the training data
of DeepL or Opus–MT) and literariness (ρ = –0.39, p < 0.05). However,
there is no such correlation between Opus-MT and the Combo system. We
do not find any significant correlation involving COMET scores and TTR.
We then explore whether n-gram overlap and literariness correlate
(Figure 1.5). They do in fact and lead to the strongest correlation so far
( ρ = –0.76, p < 0.001). A possible explanation for this is that literary works
tend to contain more creative and original language, meaning that it is
Figure 1.5 L
inear correlation between n-gram overlap of our training and test set
and predicted literariness of the human translations.
more likely that n-grams contained in the literary works have not been
seen in the training data.
1.5 Human Evaluation
We performed a human evaluation of almost every system built in this
project in order to track the effects of the system’s development and to
compare the human evaluation to the BLEU and COMET scores in the
automatic evaluation, particularly in the case of the final, combined sys-
tem. Below we describe how we conducted two of the human evaluation
experiments and their results.
1.5.1 S3Big vs S3BigDoc800: Sentence-Level vs Document-Level
At this point in our project, the two automatic evaluation metrics, BLEU
and COMET, started to behave differently for systems trained on isolated
sentences (S3Big) and on chunks of consecutive sentences (S3BigDoc800).
Therefore, we performed a human evaluation to compare this to the pic-
ture presented by the automatic metrics (cf. Way (2018) for more on the
relative merits of both evaluation types).
In this step, a professional translator blindly compared the Dutch trans-
lations from systems S3Big and S3BigDoc800. The source text was the first
page (approx. 380 words) of Kurt Vonnegut’s short story ‘2BR02B’. They
analysed the following aspects: the number of differences between the two
translations and which translation was preferred; which error category
the non-preferred translations fell into (meaning, style, lexicon, gram-
mar, cohesion; roughly based on Short and Leech, 2007), and whether the
errors concerned accuracy or fluency. Our hypothesis was that the system
trained on chunks of consecutive sentences would perform better in terms
of cohesion.
The two translations show 26 differences; in four instances, the transla-
tion choices are considered equal. Thirteen translations by the sentence-
level system are preferred, and nine by the document-level system, i.e. the
sentence-level system shows fewer issues. The issues are distributed as
shown in Table 1.3.
We compared the systems using the TAUS Dynamic Quality Framework
(Görög, 2014), an error typology often used in MT evaluation research to
distinguish accuracy (in terms of transferring meaning) and fluency errors.
The document-level system shows six accuracy (five are serious mistransla-
tions) and seven fluency issues. The sentence-level system shows three accu-
racy and five fluency issues, plus one locale convention issue. Most notably,
the document-level system more often interprets the original wrongly (5
vs 2), once leading to a completely unintelligible translation. The docu-
ment-level system produces five lexical issues, but the sentence-level system
Table 1.3 Distribution of translation issues in systems

S3Big and S3BigDoc800.
S3Big (sent-level) S3BigDoc800

(doc-level)
Meaning 2 5
Style 3 1
Lexicon 0 5
Grammar 1 1
Cohesion 3 1
Total: 9 13
generates none. However, the document-level system shows fewer style

and, perhaps even more importantly, cohesion issues (1 vs 3).
Here is an example of more cohesion in the document-level
(S3BigDoc800) system:
Everything was perfectly swell. (…) All diseases were conquered.
[S3Big] Alles was volmaakt in orde. (…) Alle ziekten werden

overwonnen.
[S3BigDoc800] Alles was perfect in orde. (…) Alle ziekten waren

overwonnen.
>Explanation: werden [S3Big] describes a process, whereas waren (like

was) [S3BigDoc800] describes a state, just like was and were in the
source text).
Here is an example of a mistranslation in the document-level system:
Wehling was fifty-six, a mere stripling in a population whose average

age was one hundred and twenty-nine.
[S3Big] Wehling was zesenvijftig, niet meer dan een groentje in een
bevolking van gemiddeld honderdnegenentwintig jaar.
[S3BigDoc800] Wehling was zesenvijftig, niet meer dan een gestreepte

populatie in een bevolking van gemiddeld honderdnegenentwintig jaar.
>Explanation: gestreepte populatie [S3BigDoc800] back-translates as

striped population, which is nonsensical here; groentje [S3Big], like
stripling, is a metaphor for a young (‘green’), inexperienced person.
1.5.2 Combined System vs DeepL
The final stage of our study led to the hypothesis that combining MT
systems that are trained on instances with different granularity (single
sentences and multiple consecutive sentences) can lead to better output
than a generic MT system. We performed an elaborate survey evaluation
to see whether the human evaluation would complement the automatic
evaluation.
In the survey, we asked respondents to compare two different Dutch
translations, one generated by DeepL and one by our Combo system, of an
English self-contained excerpt (approx. 400–600 words) from one of the
following four novels:
1. Thomas Pynchon, Gravity’s Rainbow (1973)

2. Irvin D. Yalom, The Spinoza Problem: A Novel (2012)
3. Patricia Highsmith, Ripley Under Water (1999)
4. J.D. Salinger, The Catcher in the Rye (1951)
In half of the PDF files (containing the original excerpt plus the two trans-
lations) respondents received, we swapped the order of the two transla-
tions so as to avoid order bias. The translations were labelled [translation
1] and [translation 2] so as to ensure a blind comparison.
We selected these four novels from our test set to represent literary fic-
tion, though with different degrees of literariness (see the HT predictions
in Figure 1.2). From these novels, we selected three types of excerpts that
we deemed would lead to significant insights: (a) the opening passage,
(b) a passage from the chunk with the highest predicted literariness (see
Section 1.4.1) for the human translation (which was not included in the
survey), and (c) a passage from the chunk with the largest difference in
predicted literariness between the human translation and the average score
for all of the MT systems. The latter choice reflects the idea that, this way,
our combined system would not be favoured.
In the survey, we asked respondents to rank the two translations and
motivate their ranking using examples. After that, we asked respondents
to indicate to what extent meaning, style, grammar, and cohesion were
preserved in each of the two translations. In addition, we asked them to
provide examples to substantiate their opinion with each of the four cate-
gories. Lastly, respondents were asked how literary they found each trans-
lation to be on a scale of 1 to 7, again complemented with the request to
provide examples as to why.
Nineteen respondents completed our survey. Sixteen respondents iden-
tify as female (three as male); all respondents are aged over 35 and have
completed higher education. Sixteen indicated they read for pleasure, and
8 are professional translators. Out of 19, 17 are native speakers of Dutch

(one of English, one of Spanish). This population is not very diverse but
does represent the average Dutch reader of fiction according to Leesmonitor
2015 (Stichting Lezen7): female, over 35 and with higher education.
The 19 respondents are distributed as follows: five read Highsmith, four
Pynchon, four Salinger, and six Yalom. Nine respondents read opening
passages (a), seven read passages with high literariness (b), and three read
passages with the largest difference in literariness between the HT and our
MT systems on average. This means the latter category is underrepresented
in our results.
The results are shown in Table 1.4. All mean scores for both systems are
above average. Five out of 19 respondents prefer Combo, while 14 prefer
DeepL. For the different aspects (meaning, style, grammar, cohesion) of
the translations, the respondents also favour DeepL, although only for
cohesion is the difference significant. Notable are the high DeepL scores
for literariness, and that both systems perform better on grammar and
cohesion than on meaning and style.
We also wanted to investigate whether the types of excerpts (a), (b),
and (c) would show relevant differences in scores. We left the (c) excerpts
out of our analysis, as too few respondents (just 3) completed the survey
for these.
In opening passages (Table 1.5), six respondents preferred the DeepL
translations and three our Combo output. Only once, for grammar, does
Combo perform better; the DeepL output performs better for all other
aspects. Notable are the positive evaluation of grammar and cohesion in
both translations, and the low literariness score for the Combo translation.
Table 1.4 Mean scores from the overall survey (N = 19).
Combo (std. DeepL (std.

dev.) dev.)
Meaning 3.2 (1.2) 3.8 (0.79)
Style 3.0 (1.05) 3.6 (0.84)
Grammar 3.7 (0.95) 4.2 (0.67)
Cohesion 3.4 (1.1) 4.3 (0.83)*
Literariness (1–7) 3.9 (1.5) 5.1 (1.4)*
* indicates a significant difference (paired t–test, p<0.05). The

standard deviation (Field et al., 2012, p. 37) is indicated in
parentheses and reflects how spread out the scores are.
7 https://www.lezen.nl/sites/default/files/Leesmonitor_lr.pdf.
Table 1.5 Mean scores for opening passages (N = 9).
Combo DeepL
Meaning 3.0 (1.4) 4.0 (0.82)
Style 3.0 (0.82) 3.8 (0.96)
Grammar 4.0 (0.82) 4.3 (0.58)
Cohesion 3.7 (1.5) 4.3 (0.58)
Literariness (1–7) 2.7 (1.5) 4.7 (2.3)
Table 1.6 Mean scores for high literary passages

(N = 7).
Combo DeepL
Meaning 3.4 (1.3) 3.6 (0.89)
Style 3.2 (1.3) 3.4 (0.89)
Grammar 3.4 (1.1) 4.0 (0.71)
Cohesion 3.4 (1.1) 4.4 (0.89)
Literariness (1–7) 4.4 (1.3) 5.2 (0.84)
In high literariness passages (Table 1.6), six respondents preferred the

DeepL translations, and only one the Combo output. It is notable that all
scores for the DeepL translation are lower than for the opening passages,
except those for cohesion and literariness. In addition, the Combo output
performs substantially better in terms of literariness than in the opening
passages.
When asked to motivate their ranking, respondents’ comments fall into
four lines of argumentation (which are related to each other): they find the
translations either (1) correct or incorrect, (2) literal or free, (3) faithful or
unfaithful to the original, and (4) natural/fluent or unnatural/not fluent.
(1) Several respondents find that the Combo translations contain ‘wrong
choices’, ‘errors’, ‘omissions’, and ‘mistranslations’, whereas the DeepL
translations contain fewer of these and are ‘more correct in word
choice and punctuation’. One respondent indicates they are ‘bothered
by the many wrong choices’ in the Combo translation. However, one
respondent disagrees and finds that in both translations ‘word choice is
wrong and too literal’, particularly in the DeepL translation.
(2) Several respondents find that the Combo translation is ‘too literal’.
One of them asks whether it was ‘perhaps done by Google Translate?’.
Another respondent indicates they ‘could hear the English original’ in
the Combo translation, and that they found the DeepL translation
‘more active’. Lastly, one respondent finds ‘both translations are very
bad and literal and do not take Dutch into account, at the sentence as
well as the word level’.
(3) Dovetailing with these comments on source-language interference,
respondents also comment on the translations’ relationship with the
original. Several indicate that ‘the atmosphere and ‘way of speak-
ing’ dovetail better with the original’ in the DeepL translation, and
that it is ‘better in line with what the author wants to convey’. One
respondent disagrees, however, indicating that word choice in the
Combo translation ‘is closer to the original, causing its atmosphere to
be closer to the original too’ (in their case ‘equally pompous’).
(4) Many respondents refer to the fluency of both translations, prefer-
ring the DeepL translation: it is considered ‘more idiomatic’ (twice)
and ‘more natural’. Surprisingly, in terms of fluency, two respondents
indicate that in the DeepL translation ‘sentences do not always flow’
and that in the Combo translation ‘sentences flow better’. However,
two other respondents state that the Combo translation is ‘less fluent’
than the DeepL one, which ‘reads like better Dutch’.
After presenting the respondents’ evaluations in general terms, we would

like to draw attention to notable examples that were mentioned by several
respondents.
The Combo system omitted to translate two short sentences. It also
ignored stylistic repetition in at least two instances: ‘finally finally’ was
translated as ‘eindelijk’ (finally) by Combo, and as ‘eindelijk eindelijk’
(finally finally) by DeepL. Another example of this is ‘insert coins insert
coins insert coins’, which was translated as ‘stopte munten in munten’
(inserted coins in coins) (a mistranslation) by Combo and as ‘munten
inwerpen munten inwerpen munten inwerpen’ (insert coins insert coins
insert coins) by DeepL.
According to the respondents, Yalom’s novel is characterised by a
‘pompous’ or ‘haughty’ register. Its translation clearly is a topic of dis-
cussion: several respondents refer to ‘private talk’, which is translated as
‘privégesprek’ (private conversation) by DeepL, and as ‘gesprek onder vier
ogen’ (conversation under four eyes) by Combo. The latter is considered
an ‘exaggerated choice’. However, several respondents also defend such
‘exaggerated’ word choices as they are closer to the original, preferring, for
example, ‘volkomen gelijkwaardig (completely) (Combo) vs ‘volledig geli-
jkwaardig’ (fully) (DeepL) and ‘die de wereld waarlijk veranderen’ (verily)
(Combo) vs ‘die de wereld werkelijk veranderen’ (truly) (DeepL). In more
general terms, some respondents find that the original’s ‘register is deeply
wrong’ in the DeepL translation, and ‘even more exaggerated’ in the
Combo translation, its pompousness ‘going overboard’. One respondent
sticks to their guns, stating that ‘the pompous language in the original
serves a purpose’, which the Combo output ‘serves better’.
Remarkably, when asked about grammaticality, all but one of the exam-
ples which respondents provided relate to verbs, more specifically to tenses,
collocations, and source-language interference. Two examples where the
DeepL translation is preferred are related to tenses: ‘had always fascinated
me’ translated as ‘had me altijd gefascineerd’ (had fascinated; note that ‘had’
in the translation is an OOV, despite being quite a common English word)
by Combo, and as ‘heeft mij altijd gefascineerd’ (has fascinated) by DeepL,
and the Combo translation ‘en vroeg wie er behalve ik ging’ (went) (correct
would be ‘zou gaan’ (would go)). An example of a wrong collocation can
be found in the Combo output ‘Hij heeft nu een hoop geld. Dat deed hij
vroeger niet’ (He has lots of money now. Before he hadn’t.). Respondents
seem to be quite satisfied with the level of cohesion of both translations,
which is reflected in the scores above (3.4 for Combo vs 4.4 for DeepL)
and supported by comments such as ‘an important ‘break’ in the original is
transferred well to both translations’, ‘both translations are rather cohesive’
or ‘equally cohesive as the original’. Two respondents clearly preferred the
DeepL translation: it has a ‘better rhythm’, whereas the Combo output ‘is
incohesive’ and ‘sometimes sounds like a bullet-pointed list, not a story (but
both cover the content of the original)’.
When asked for the motivation of their literariness scores, two respond-
ents clearly indicated they needed a definition of ‘literary’: ‘I don’t under-
stand the meaning of literary’ and ‘please define literary’. One respondent
indicated they found both translations ‘not literary because ‘nothing was
done’ with the original, it was not turned into a stand-alone or let alone
Dutch version, but they are word-for-word translations based on the first
dictionary available’.
We decided to reveal the survey’s objective and the ‘translators’ to the
respondents only after they had completed the survey: ‘With this survey
we aim to gain an insight into (1) whether a translation machine built for
literature performs better than a generic one, and (2) to what extent the
output of these translation machine types is conceived as literary’. We also
revealed to them which translation came from the literary-adapted system
and which from the ‘generic’ one, and invited them to leave a reaction. We
want to include two responses here: ‘It’s funny, I thought the first transla-
tion [Combo] was made by a machine and the second [DeepL] by a bad
translator’. The other respondent ‘instinctively’ did not like the first trans-
lation [Combo] but was ‘still surprised it was made by a machine’.
1.6 Conclusions and Future Work

We built literary-adapted sentence- and document-level MT systems for
English-to-Dutch using larger amounts of in-domain training data than
in previous work and have evaluated them against two types of generic
state-of-the-art MT systems: commercial (DeepL) and open-source

(Opus-MT). Returning to our original research questions, we find:
RQ1. The automatic evaluation shows that our best literary-adapted MT

system outperforms a generic open-source system by a wide mar-
gin in all cases, and it performs better than a commercial online
system in most cases, especially so for genre fiction. However,
DeepL comes slightly ahead of ours on some of the highly literary
novels (Figure 1.4). This is even more pronounced in the human
evaluation: for all four literary novels, DeepL is preferred over our
system (Section 1.5.2). In addition, the human evaluation reveals
clues that both MT outputs score better on grammar and cohesion
than on meaning and style. Grammar issues mostly concern verbs,
in particular tenses. Lastly, where ‘literary’ vocabulary (according
to survey respondents) is used, opinions diverge widely on how
this should be translated (more accessible vs maintaining style).
RQ2. According to the automatic evaluation metric COMET, docu-
ment-level MT performs slightly better than sentence-level and a
combination of the two leads to further improvement, revealing
that they may be complementary. This is an interesting finding
as most if not all people working on this topic nowadays con-
trast one against the other, but do not consider ensembling the
two. The human evaluation hints that document-level MT leads
to increased cohesion but also to a less precise transfer of meaning
at the lexical level.
RQ3. Both in terms of literariness and lexical variety, the human
translations tend to score highest, followed by DeepL, our best
system, and Opus-MT. Correlations show that the higher the
predicted literariness of a novel, the less of an advantage (in
terms of automatic evaluation score) our system has over DeepL.
A possible explanation could be that DeepL has almost certainly
been trained on a wider variety of general-domain data than our
system, which may give DeepL an edge when it comes to general-
ising. However, that does not seem to be the case since there is no
correlation between Opus-MT (also trained on general-domain
data) and our system. Another possible explanation is that our
system may have overfitted towards genre fiction, which predom-
inates in the training data rather than literary novels. Finally, we
can speculate about other possible reasons for DeepL’s edge on
highly literary novels, such as a very large network and/or a mul-
tilingual model. That said, we cannot know for sure since there
is no information available about how the system is built. The
human evaluation shows that both MT outputs of high-literary
passages are appreciated less (i.e. score worse) than other, less
literary passages.
RQ4. Our findings provide support for the validity of the predictive
literariness model. Translations by DeepL tend to have slightly
higher predicted literariness than our Combo system, and for most
novels, the human translation is predicted as most literary. When
presented with fragments of four novels, the survey respondents
agreed with this ranking of DeepL and our Combo system.
Finally, we outline two main directions for future work. First, we note that
literary adaptation provides the biggest edge over DeepL on genre fiction,
even if the adaptation followed is rather general, for fiction as a whole. It
may, therefore, be better to conduct a more fine-grained adaptation at the
level of genre (e.g. suspense) or even at the author and/or translator level,
although, of course, the more fine-grained the experiment, the less data will
be available. Second, we found that the advantage of the literary-adapted
system tends to be smaller for novels with higher predicted literariness, and
in general, produces translations with lower predicted literariness and lexi-
cal variety. Methodologies to foster more natural and varied output could
be explored (Freitag et al., 2022). It may also be promising to modify the
MT system so that it prefers more literary outputs. Finally, multilingual
models and larger networks may lead to MT systems that generalise better.
Acknowledgements
We would like to thank the Center for Information Technology of the
University of Groningen for their support and for providing access to the
Peregrine high-performance computing cluster. We are grateful to Karina
van Dalen-Oskam and the reviewers for their helpful feedback.
References
Baas, M. (2021). Context is key: The effects of contextual information in English–
Dutch neural machine translation for different domains [Unpublished master’s
thesis]. University of Groningen. https://arts.studenttheses.ub.rug.nl/29168/
Baldick, C. (2008). Literariness. In The Oxford dictionary of literary terms (3rd
edn.). Oxford University Press. https://oxfordreference.com/view/10.1093/acref
/9780199208272.001.0001/acref-9780199208272-e-661
Baldick, C. (2015). Genre fiction. In The Oxford dictionary of literary terms (4th
edn.). Oxford University Press. https://oxfordreference.com/view/10.1093/acref
/9780198715443.001.0001/acref-9780198715443-e-494
Boyne, J. (2006). The boy in the striped pyjamas. David Fickling Books.
Boyne, J. (2010). De jongen in de gestreepte pyjama (J. de Jonge, Trans.). Boekerij
(Original work Boyne, 2006).
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edn.).
Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203771587
Field, A., Miles, J., & Field, Z. (2012). Discovering statistics using R. Sage
Publications. https://nyu-cdsc.github.io/learningr/assets/discoveringstatistics.pdf
Freitag, M., Vilar, D., Grangier, D., Cherry, C., & Foster, G. (2022). A natural
diet: Towards improving naturalness of machine translation output. In Findings
of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland
(pp. 3340–53). Association for Computational Linguistics. https://aclanthology
.org/2022.findings-acl.263/
Görög, A. (2014). Quality evaluation today: The dynamic quality framework. In
Proceedings of Translating and the Computer, 36, 155–64. https://aclanthology
.org/2014.tc-1.21.pdf
Junczys-Dowmunt, M., Grundkiewicz, R., Dwojak, T., Hoang, H., Heafield, K.,
Neckermann, T., Seide, F., Germann, U., Fikri Aji, A., Bogoychev, N., Martins,
A.F.T., & Birch, A. (2018). Marian: Fast neural machine translation in C++. In
Proceedings of ACL 2018, System Demonstrations, Melbourne, Australia (pp.
116–21). Association for Computational Linguistics. https://aclanthology.org/
P18-4020/
Kocmi, T., Federmann, C., Grundkiewicz, R., Junczys-Dowmunt, M., Matsushita,
H., & Menezes, A. (2021). To ship or not to ship: An extensive evaluation
of automatic metrics for machine translation. In Proceedings of the Sixth
Conference on Machine Translation, online (pp. 478–94). Association for
Computational Linguistics. https://aclanthology.org/2021.wmt-1.57/
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N.,
Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin,
A., & Herbst, E. (2007). Moses: Open source toolkit for statistical machine
translation. In Proceedings of the 45th Annual Meeting of the Association for
Computational Linguistics Companion Volume Proceedings of the Demo and
Poster Sessions, Prague, Czech Republic (pp. 177–80). https://aclanthology.org
/P07-2045/
Koolen, C., van Dalen-Oskam, K., van Cranenburgh, A., & Nagelhout, E. (2020).
Literary quality in the eye of the Dutch reader: The national reader survey.
Poetics, 79. https://doi.org/10.1016/j.poetic.2020.101439
Kudo, T., & Richardson, J. (2018). SentencePiece: A simple and language
independent subword tokenizer and detokenizer for neural text processing. In
Proceedings of the 2018 Conference on Empirical Methods in Natural Language
Processing: System Demonstrations, Brussels, Belgium (pp. 66–71). Association
for Computational Linguistics. https://aclanthology.org/D18-2012/
XVII: The Qualities of Literary Machine Translation, Dublin, Ireland (pp. 1–9).
-7301.pdf
Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and
documents. In Proceedings of the 31st International Conference on Machine
Learning, Beijing, China (pp. 1188–96). https://doi.org/10.48550/arXiv.1405
.4053

Association for Machine Translation. https://aclanthology.org/W19-7302/
Oliver, A., Toral, A & Guerberof-Arenas, A. (2022). Bilingual e-books via neural
machine translation and their reception. In J.L. Hadley, K. Taivalkoski-
Shilov, C.S.C. Teixeira, & A. Toral (Eds.), Using technologies for creative-text
translation (pp. 89–115). Routledge. https://doi.org/10.4324/9781003094159
meeting of the Association for Computational Linguistics, Philadelphia, PA (pp.
P02-1040/
Poncelas, A., Shterionov, D., Way, A., Maillette de Buy Wenniger, G., & Passban,
P. (2018). Investigating backtranslation in neural machine translation. In
Proceedings of the 21st Annual Conference of the European Association for
Machine Translation (EAMT 2018), Alicante, Spain (pp. 249–58). https://doi
.org/10.48550/arXiv.1804.0618
Popel, M,, Macháček, D., Auersperger, M., Bojar, O., & Pecina, P. (2019). English-
Czech systems in WMT19: Document-level transformer. In Proceedings of the
Fourth Conference on Machine Translation (Volume 2: Shared Task Papers,
Day 1), Florence, Italy (pp. 342–48). Association for Computational Linguistics.
https://aclanthology.org/W19-5337/
Rei, R., Stewart, C., Farinha, A.C., & Lavie, A. (2020). COMET: A neural
framework for MT evaluation. In Proceedings of the 2020 Conference on
Empirical Methods in Natural Language Processing (EMNLP), online (pp.
2685–702). Association for Computational Linguistics. https://aclanthology
.org/2020.emnlp-main.213/
Sennrich, R., Haddow, B., & Birch, A. (2016). Improving neural machine
translation models with monolingual data. In Proceedings of the 54th Annual
Meeting of the Association for Computational Linguistics (Volume 1: Long
Papers), Berlin, Germany (pp. 86–96). https://aclanthology.org/P16-1009/
Short, M., & Leech, G. (2007). A method of analysis and some examples. In M.
Short & G. Leech (Ed.), Style in fiction: A linguistic introduction to English
fictional prose (pp. 60–66). Routledge. http://sv-etc.nl/styleinfiction.pdf
Squires, C. (2009). Marketing literature: The making of contemporary writing in
Britain. Palgrave Macmillan.
Thompson, B., & Koehn, P. (2019). Vecalign: Improved sentence alignment in
linear time and space. In Proceedings of the 2019 Conference on Empirical
Methods in Natural Language Processing and the 9th International Joint
Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong,
China (pp. 1342–48). Association for Computational Linguistics. https://
aclanthology.org/D19-1136/
Tiedemann, J., & Thottingal, S. (2020). Opus-MT — Building open translation
services for the World. In Proceedings of the 22nd Annual Conference of the
European Association for Machine Translation, Lisbon, Portugal (pp. 479–80).
European Association for Machine Translation. https://aclanthology.org/2020
.eamt-1.61/
Toral, A., Oliver, A., & Ribas Ballestín, P. (2020). Machine translation of novels
in the age of transformer. In J. Porsiel (Ed.), Maschinelle Übersetzung für
Übersetzungsprofis (pp. 276–95). BDÜ Fachverlag. https://doi.org/10.48550/
arXiv.2011.14979
Van Cranenburgh, A., Van Dalen-Oskam, K., & Van Zundert, J. (2019). Vector
space explorations of literary language. Language Resources & Evaluation 53,
625–50. https://www.researchgate.net/publication/330990258_Vector_space
_explorations_of_literary_language
Van Noord, G. (2006). At last parsing is now operational. In Verbum Ex Machina.
Actes de la 13e conférence sur le traitement automatique des langues naturelles,
Leuven, Belgium (pp. 20–42). https://aclanthology.org/2006.jeptalnrecital
-invite.2/
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, Ll, Gomez, A.N., Kaiser,
L., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural
Information Processing Systems 30: Annual Conference on Neural Information
Processing Systems 2017, Long Beach, CA (pp. 5998–6008). https://doi.org/10
.48550/arXiv.1706.03762
Voigt, R., & Jurafsky, D. (2012). Towards a literary machine translation: The
role of referential cohesion. In D. Elson, A. Kazantseva, R. Mihalcea, & S.
Szpakowicz (Eds.), Proceedings of the NAACL-HLT 2012 Workshop on
Computational Linguistics for Literature (pp. 18–25). Association for
Computational Linguistics. https://aclanthology.org/W12-25.pdf
.1803.08409
Appendix A: Test Set Novels
Author Title Year original Year translation

Auster, Paul Sunset Park 2010 2015
Baldacci, David Divine justice 2008 2008
Barnes, Julian The Sense of an Ending 2011 2011
Carre, John le Our Kind of Traitor 2010 2010
Clancy, Tom Locked On 2011 2012
Franzen, Jonathan The Corrections 2001 2001
French, Nicci Blue Monday 2011 2011
Golding, William Lord of the Flies 1954 1962
Grisham, John The Confession 2010 2010
Hemingway, Ernest The Old Man and the 1951 2009
Sea
Highsmith, Patricia Ripley under Water 1991 1992
Hosseini, Khaled A Thousand Splendid 2007 2010
Suns
Irving, John Last Night in Twisted 2009 2010
River
James, E.L. Fifty Shades of Grey 2011 2012
Joyce, James Ulysses 1922 2012
Kerouac, Jack On the Road 1957 1988
King, Stephen 11/22/63 2011 2011
Kinsella, Sophie Shopaholic and Baby 2007 2007
Mitchell, David The Thousand Autumns 2010 2010
of Jacob de Zoet
Orwell, George 1984 1949 1984
Patterson, James The Quickie 2007 2008
Pynchon, Thomas Gravity's Rainbow 1973 1992
Roth, Philip The Plot against 2004 2004
America
Rowling, J.K. Harry Potter and the 2007 2007
Deathly Hallows
Salinger, J.D. Catcher in the Rye 1951 1989
Slaughter, Karin Fractured 2008 2008
Steinbeck, John The Grapes of Wrath 1939 1940
Tolkien, J.R.R. LotR: The Return of 1955 1977
the King
Twain, Mark Adventures of 1884 1997
Huckleberry Finn
Wilde, Oscar The Picture of Dorian 1891 1919
Gray
Yalom, Irvin D. The Spinoza Problem 2012 2012
2 ‘I Am a Bit Surprised’
Literary Translation and Post-Editing
Processes Compared
Waltraud Kolb
2.1 Introduction
At the International Literature Festival in Berlin in 2019, a panel com-
posed of two authors/translators and a computer linguist1 discussed the
future role of machine translation (MT) in the literary domain. It was a
remarkable early effort to draw the reading public’s attention to recent
developments in the field. Towards the end of the event, moderator Gregor
Dotzauer, a literary editor at the German newspaper Tagesspiegel, sin-
gled out Ernest Hemingway as an obvious candidate for MT because he
‘has this Latin clarity’ (Internationales Literaturfestival Berlin, 2019).
Hemingway is indeed well known for his straightforward prose, and one
of his early short stories also serves as the source text for the explorations
of literary post-editing that will be presented in this chapter. Post-editing
lies at the very heart of any investigation of the opportunities and chal-
lenges engendered by digital technologies in the literary translation domain
as MT is still widely considered ‘unsuitable for translating literature’
(European Commission, Directorate-General for Education, Youth, Sport
and Culture, 2022, p. 60), unless followed by human post-editing. So far,
we do not know very much about what post-editing actually entails for
literary translation, be it literary translation as a process or as a product.
The aim of this chapter is to examine the differences between translation
from scratch and post-editing processes in the literary domain from a
primarily cognitive perspective and identify correlations with the resulting
target texts. In which ways is post-editing a short story by Hemingway
different from translating it from scratch? Is it faster, does it involve
less typing? Do post-editors reflect on the same issues as translators and
go through the same decision-making routines? Do both engage in the
same way with Hemingway’s choices? Are post-editors primed by MT
suggestions? To explore these questions, I will compare findings from two
1 Karin Tidbeck, Sylvain Neuvel, and Aljoscha Burchardt.
DOI: 10.4324/9781003357391-4
54 Waltraud Kolb
empirical studies: in the first study, five professional literary translators

translated Hemingway’s story from scratch and their working and decision-
making processes were captured by keylogging and think-aloud (Kolb,
2011, 2013, 2017, 2021); in the second study, using the same research
design, five different professional literary translators post-edited a raw MT
version of the same short story generated by DeepL.2
2.2 Related Research
Post-editing (PE) is ‘the activity of revising a text that has been translated
automatically by a Machine Translation (MT) system’ (O’Brien, 2021,
p. 177), usually with the aim of increasing productivity. In this context,
productivity can be measured as the effort that goes into post-editing a
machine-translated text, compared to the effort that goes into translating
the same text from scratch, with lower effort signifying higher productivity.
Krings (2001) proposed three levels to study PE effort that have since been
widely applied in research, namely the temporal level (time spent on the
task), the technical level (number of keystrokes), and the cognitive level
(e.g. number and duration of pauses). Regarding literary texts, Moorkens
et al. (2018) and Toral et al. (2018), reporting on the same experiment,
found substantial productivity gains for PE (for both statistical MT and
neural MT) compared to human translation (HT) for their (English–
Catalan) literary-adapted MT systems. Their participants, six professional
translators with experience in literary translation, were faster post-editing
than translating, generating fewer keystrokes, and PE resulted in fewer but
longer pauses (see Moorkens et al., 2018, for the participants’ perception
of the task). The same parameters were also used in a later study by
Guerberof-Arenas and Toral (2022) involving the language pairs English–
Catalan and English–Dutch with four literary translators as participants
(see Chapter 1, this volume); their results were mixed, with a lower
technical effort for PE for both language pairs but a higher temporal effort
for PE compared to HT for Dutch. For both target languages, the average
number of pauses was also lower for PE than for HT, which the authors
interpreted as an indicator of lower cognitive effort, adding, however, that
the higher cognitive effort in HT might indicate that translators ‘“think
harder” for a more creative solution’ (p. 13). Both studies used keylogging
as a tool and based their analysis of cognitive effort on pause patterns.
In order to gain more insight into the cognitive aspects of translatorial
processes as well as what happens during pauses and how translators and
post-editors make their decisions, Krings (2001) and Vieira (2015)—both
2 www.deepl.com/en/translator.
‘I Am a Bit Surprised’ 55
dealing with non-literary texts—used think-aloud protocols (TAPs). In the

literary domain, TAPs have occasionally been used to explore HT processes
(Jones, 2011; Kolb, 2011, 2013, 2017, 2021; Borg, 2022), but, to the best
of my knowledge, the present study is the first to use such verbal data for
the study of literary PE processes.
Effort and productivity need to be assessed against questions of quality.
Regarding non-literary texts, numerous studies have found PE to produce
comparable, or sometimes even better, quality than HT (Koponen, 2016;
Daems et al., 2017) A central issue in exploring PE quality is the ques-
tion of whether post-edited texts exhibit characteristics of ‘post-editese’
(see Section I.3, Introduction, this volume), a term introduced by Daems
et al. (2017) to describe ‘the expected unique characteristics of a PE text
that set it apart from a translated text’ (p. 90). So far, research has yielded
mixed results. While Daems et al. (2017) have not found any indication of
post-editese in their study of (English–Dutch) translated and post-edited
newspaper articles, a number of other studies did identify post-editese in
PE output. Čulo and Nitzke (2016) compared the two modes in terms of
terminology (English–German) and found that there was less variation in
PE than in HT. Farrell (2018) also found lower variation in PE as well
as normalization and homogenization phenomena (English–Italian). In
a study involving five languages (English, German, French, Spanish, and
Chinese), Toral (2019) also reported normalization of the target language
in PE, in addition to simplification and more interference from the original
than in HT, so that he even speaks of post-editese as ‘exacerbated trans-
lationese’ (p. 273). Volkart and Bouillon (2022) confirmed some of these
features for an authentic corpus of human-translated and post-edited press
releases issued by the European Investment Bank.
Regarding literary PE, Castilho et al. (2019) compared the presence
of post-editese in the news and literary domains for the language-pair
English–Brazilian Portuguese and found the level of post-editese to
be higher for literature. In a subsequent study, Castilho and Resende
(2022a) analyzed features of post-editese in literary texts, using a range of
parameters such as lexical richness (see e.g. Vanmassenhove et al. (2019)
on the loss of lexical diversity in MT), lexical density, sentence length,
punctuation, explicitation, personal pronouns, and convergence between
modalities. They found that PE output was similar to MT output in terms
of lexical density, use of pronouns, and sentence length, with mixed
results for the other categories, also depending on the type of literary text.
They interpreted their results as an indication of priming effects through
MT (see also Castilho and Resende, 2022b). That MT priming has a
substantial impact on a post-editor’s textual voice has also been shown by
Kenny and Winters (2020; see also Chapter 3 in the present volume). They
asked Hans-Christian Oeser, a well-known German translator of English
56 Waltraud Kolb
literature, to post-edit a chapter of a novel that he had translated from

scratch two decades earlier and found that his voice—as reflected in his
stylistic choices—is less manifest in his post-edited text than in his HT.
Macken et al. (2022) designed a three-stage study in which an English
source text was first machine-translated into Dutch, the raw MT was then
post-edited by a professional literary translator, and, as a third step, this
version was revised (mainly monolingually) by a different translator. The
results show that the MT and the post-edited version were more similar
to each other than the post-edited version and the revision, and that more
editing occurred during revision than during PE, with the reviser’s edits
primarily aiming at improving readability and acceptability for target
readers.
Guerberof-Arenas and Toral (2020) measured creativity as an indicator
of quality in HT, MT, and PE for the language pair English–Catalan,
with two professional literary translators as participants. To determine
creativity, they looked at acceptability (number of errors) and novelty
(creative shifts). Regarding acceptability, they found that PE scored best
for accuracy and HT for fluency; regarding novelty, HT scored higher than
PE, indicating that PE is constrained by the MT output. In a subsequent
study, Guerberof-Arenas and Toral (2022) used a similar design but
added the language pair English–Dutch. They describe their results as
a ‘cautionary tale’ for using MT, ‘because the translator becomes the
evaluator and not the creator’ (p. 207). That the post-editor’s role as
evaluator (see also Chapter 7 in the present volume) has a profound
impact on decision-making processes in PE will also become apparent
below.
2.3 Research Design
In a study of literary translation processes carried out in 2009/2010 (Study
1), five literary translators (four female, one male; henceforth referenced as
T–1 to T–5) translated a short story by Ernest Hemingway into German.
Their processes were captured by keylogging and think-aloud, product data
comprised first drafts, interim, and final versions; information about the
participants’ educational and professional backgrounds and work routines
was collected through questionnaires (Kolb, 2011, 2013, 2017, 2021). To
explore literary PE and compare it to HT, a follow-up study was carried
out in 2020/2021 (Study 2), in which five different literary translators
(all female: PE–1 to PE–5) post-edited a DeepL version of the same story,
using the same type of tools. DeepL was selected as a general-domain
neural MT system as it is widely used in German-speaking countries and
would, in practice, be a likely choice for literary translators (who would
usually not be in a position to train their own MT systems; see Section I.2,
Introduction, this volume, for more on MT system training, tuning, and

testing).
The participants of both groups were experienced literary translators
who had been working as such between eight and 29 years at the time of
the respective study. It is important to note that none of the participants in
Study 2 had any previous post-editing experience, which is still typical of
the literary translation field at this point. The participants of both groups
worked in their usual work environments: in most cases, a space in their
homes, and in three cases an office away from home. They were asked to
follow their usual work routines as far as possible and work on the target
text until they felt it would be ready to be sent to a publisher. Both groups
were asked to think aloud as much as possible while working on their task,
verbalizing their thoughts and deliberations; Audacity3 was used as the
voice-recording software. They were also asked to record retrospectively
any thoughts they had while they were not at their desk but, for example,
were loading the dishwasher. To capture their keystrokes, Group 1 used
Translog (Jakobsen & Schou, 1999), and Group 2 Inputlog (Leijten &
Van Waes, 2013). Inputlog was used in Study 2 because some participants
in Group 1 had experienced technical problems with Translog. Both tools
log the same type of data of interest, most notably, insertions, deletions,
and pauses. Conditions for the participants were comparable so that the
choice of tool did not affect their work: Translog users worked in a target
text window that resembles common text editors, Inputlog users in an
MS Word window; and both tools have the advantage that texts are not
split into segments, making it easier to keep one's eye on document-level
narrative and stylistic features.
The source text was a short story by Hemingway appropriately entitled
‘A Very Short Story’ (1925) about a man and a woman who meet in a
hospital in Padua in northern Italy at the end of WWI. The unnamed male
protagonist, an American soldier, is a patient there; the female protagonist
Luz is (probably) a nurse. The story is 637 words long, including the title,
which makes it manageable in a research setting and still has the advantage
of being a complete literary piece so that the participants could take into
account document-level features. Despite Hemingway’s straightforward
language (low lexical variation, preference for simple and common words,
short, paratactic sentences), the raw MT output contained a number of
errors. The types of errors are well-known problems in MT research:
disambiguation of polysemous words, orthographically similar words,
co-reference resolution, verb tense, register, literality, and omission of
3 www.audacityteam.org/.
58 Waltraud Kolb
source text material. The purpose of this chapter, however, is not to

evaluate DeepL’s performance, but to explore PE processes.
2.4 Data and Discussion

In my analysis of HT and PE processes, I will first briefly compare the
temporal and technical effort (i.e. task time and number of keystrokes)
associated with the two modalities, and then focus on the underlying
cognitive dimension. Unlike some studies mentioned above, I will not look
at pauses but will explore decision-making processes and the role played by
MT priming. Priming effects have been shown to occur on multiple levels
(Farrell, 2018; Toral, 2019; Kenny & Winters, 2020; Castilho & Resende,
2022a); in this chapter, examples of MT priming will be discussed that
relate to terminology, interpretation, normalization, and literalism.
2.4.1 Temporal and Technical Effort
For the purposes of this study, temporal effort is measured as the total
time of the verbal record. Participants were asked to switch on Audacity
whenever they worked on the task so that the verbal record includes time
spent working at the computer (including silent periods) as well as periods
they spent on manually revising the printout of an interim target version.
It does not cover some preliminary task-related activities of unknown
length, such as setting up the tools, or periods in which they reflected on
task-related issues away from their desk, e.g. during breakfast. However,
whenever participants at a later point summarized such deliberations
for the record, these brief sequences of retrospective verbalization are
included in their total time count. On three occasions, translators put on
record their desire to pause the recording and read the target text silently;
these periods are also not included. Since the participants worked in their
authentic working environments rather than in a supervised lab, some
technical hiccups occurred which also impacted time measurements: three
participants from Group 1 (T–1, T–2, T–3) had to re-type passages that had
not been saved by Translog; these periods have been subtracted from their
total times (see Kolb, 2017 for more details) so as to not skew correlations
with the respective keystroke numbers. Given the above limitations, the
time measurements in Table 2.1 are approximate values only but still serve
to indicate general tendencies for the ten participants.
As shown in Table 2.1, the post-editors spent on average less time on
the task than the translators (approx. 24% less, a number that is slightly
lower than results from other studies, e.g. Toral et al. (2018)), with the
median being only approx. 10% lower. However, a closer look reveals
that there are significant differences between individual participants. Inter-
subject variation is particularly great in Group 1, ranging from 1 hour
Total time of verbal record and total number of keystrokes for transla-
Table 2.1
tors and post-editors, charcut for post-editors.
Part. Total time of verbal Total number of charcut

record (hrs:min) keystrokes
T–1 5:00 9750
T–2 2:22 6091
T–3 2:29 6991
T–4 3:54 10247
T–5 1:35 6379
Average 3:04 7892
Median 2:29 6991
PE–1 2:03 7931 0.2816
PE–2 2:14 3464 0.3137
PE–3 2:40 5486 0.3195
PE–4 2:55 4583 0.2577
PE–5 1:46 5787 0.3656
Average 2:20 5450 0.3076
Median 2:14 5486 0.3137
and 35 minutes (T–5) to 5 hours (T–1). The participants in Group 2 spent

between 1 hour and 46 minutes (PE–5) and 2 hours and 55 minutes (PE–4)
on the PE task, their temporal effort thus varying to a lesser extent, with
no outliers.
The total number of keystrokes was approx. 31% lower on average
for the PE modality, the median approx. 21.5%. Comparing individual
task times and keystroke numbers shows that working styles differed
significantly between participants in both groups. For instance, T–1
spent by far the longest time, but did not produce the highest number
of keystrokes; the greatest technical effort in terms of keystrokes was
T–4’s, whose total working time was more than an hour shorter than
T–1’s. In Group 2, PE–1 and PE–5 spent the shortest time but produced
the most keystrokes, and PE–4 spent the longest time, while her number
of keystrokes is in the lower range. This indicates that some participants
spent more time deliberating without typing than others, or typed and then
deleted more candidate solutions for a word or phrase than others. In all,
individual speed and working style seem to have had less of an impact in
PE than in HT.
Another measure to assess PE effort is the difference (or similarity)
between the raw MT output and the final post-edited version, which can
be determined by a number of different metrics. For the purpose of this
chapter, charcut (Lardilleux & Lepage, 2017; see also Macken et al.,
2022) was used, which is a character-based metric that counts the cost of
PE in terms of deletions, insertions, and shifts. As Table 2.1 shows, the
60 Waltraud Kolb
differences between MT and post-edited versions range from 0.2577 (PE–

4) to 0.3656 (PE–5). As above, the results reflect different working styles:
for instance, PE–4’s final target text is the most similar to the raw MT
output, even though her technical effort was not the lowest and she spent
the longest time on the task; on the other end of the spectrum is PE–5,
whose target text is the most different from the MT output, even though
she was the fastest and her number of keystrokes was not the highest.
PE–1, who did by far the most typing, submitted a target text that ranges
second in terms of similarity with the MT output.
If we look at the sentence level, the participants edited between 35 and
38 out of the story’s total of 40 sentences (87.5%–95%); even more sig-
nificantly, there was not a single sentence where all five post-editors agreed
that it could be accepted without any edits. Based on my own evaluation
of accuracy (correct content) and fluency (orthographic, grammatical, idi-
omatic correctness), 15 out of the 40 sentences (37.5%) would not have
needed any editing, a percentage that is in line with findings in other stud-
ies (e.g. Macken et al., 2022). This seems to indicate that the achievement
of accuracy and fluency is not enough for professional literary translators
to feel comfortable putting their name to the final product (see also, e.g.
Moorkens et al., 2018). What goes beyond issues of accuracy and fluency
are, for instance, cultural considerations, stylistic preferences, the interpre-
tation of ambiguous language, and considerations of loyalty towards the
source text author’s choices or the target readers.
2.4.2 Cognitive Processes
The quantitative data described above, i.e. temporal and technical effort
or inter-text similarity, are shaped by the underlying cognitive processes.
In the case of PE, these processes relate to either the source text, the raw
MT output, or the target text, whereas with HT the cognitive load is dis-
tributed between two texts only rather than three. Studies on non-literary
PE have shown that the distribution of attention tends to differ between
the two modalities, with the source text usually receiving less attention in
PE than in HT (Krings, 2001; Mesa-Lao, 2014). This is hardly surprising,
given the fact that in HT drafting the first version is usually the phase in
which translators most intensely engage with the source text, its interpreta-
tion, its style, and the author’s choices; in PE, the first draft―and with it,
the first interpretation of the source text―is supplied by the MT engine.
The following discussion of cognitive processes and the role played by
MT priming will draw on data from the participants’ verbal reports. TAPs
include various kinds of verbalization, ranging from emotional reactions
(sighing, laughing) to reading out loud snippets of source or target text
or raw MT, from disjointed sequences of thinking aloud to explanations
or rationalizations addressed to the researcher (Ericsson & Simon, 1993).
They contain directly or indirectly articulated indicators of uncertainty, i.e.

a cognitive state of indecision on the part of the translator or post-editor
and subsequent problem-solving and decision-making behavior (Angelone,
2010).
Examples 1 and 2 show in which ways the participants’ engagement with
the source and target texts differed between HT and PE, how their problem-
solving routines differed, and how the post-editors were primed by DeepL’s
suggestions. The resulting post-edited target texts are more similar to each
other than those translated from scratch. Both examples are taken from the
opening paragraph of Hemingway’s story (relevant passages in italics):
One hot evening in Padua they carried him up onto the roof and he
could look out over the top of the town. There were chimney swifts
[Example 1] in the sky. After a while it got dark and the searchlights
came out. The others went down and took the bottles with them. He
and Luz could hear them below on the balcony. Luz sat on the bed
[Example 2]. She was cool and fresh in the hot night.
(Hemingway, 1925/1930, p. 83)
Example 1 relates to terminology. The German translation of chimney

swift provided by DeepL was Schornsteinsegler, one of two correct
German terms for the bird in question (Lat. chaetura pelagica). All post-
editors retained the MT suggestion. As a first step, they all double-checked
it in a Google/Wikipedia search or an online dictionary. Three of the post-
editors were quickly satisfied that DeepL had supplied the correct German
term. Two of the post-editors (PE–1 and PE–5) realized that there was a
discrepancy between the birds’ habitat and the story’s setting, as chimney
swifts live in the Americas, but not in Europe. PE–5 quickly laughed off her
initial surprise, proceeding immediately to the next segment. PE–1 was the
only one in Group 2 who briefly reflected on the implications (translations
from the German TAPs are mine, the words in italics are English in the
original TAPs):
PE–1: it is somehow written from the point of view of an American

soldier who maybe thinks they are chimney swifts, and it fits as they
are sitting on the roof … so I think we can leave it as it is … also
because I think the image is nice
PE–2: I have to look up those Schornsteinsegler, what exactly that is,

chimney swifts in the sky, if I am honest I can’t imagine what that is in
English […] let’s see if DeepL is actually right here […] so I just put it
into Google […] chimney swift, Schornsteinsegler, aha, they are indeed
Schornsteinsegler
62 Waltraud Kolb
PE–3: these chimney swifts, Schornsteinsegler, I can’t imagine that this

is correct and therefore I now open dict.cc on my computer […] chim-
ney swifts, ok, they exist and they are indeed Schornsteinsegler, which
means this is a bird
PE–4: let’s look up chimney swifts, I have never heard Schornsteinsegler,

but this does not have to mean anything, chimney swifts, I am on dict
.cc now, let’s see […] good, so DeepL has identified this all right, this
is a bird
PE–5: should I check now if this is indeed a bird […] yes […] but I am
a bit surprised now that it flies around in Italy, even though it is an
American swift, but ok [laughs]
While the post-editors were primarily focused on verifying the terminolog-

ical correctness of the MT suggestion, the translators engaged in extended
research when drafting their first target versions. Having described these
deliberations in some detail previously (Kolb, 2021), I will simply sum-
marize salient points here. With one exception, the translators based their
decisions on either a careful reflection on the implications of the habitat/
setting discrepancy or the different images that would be evoked by dif-
ferent German options. T–1 and T–2, for instance, both spent consider-
able time weighing the pros and cons of putting an American bird into
an Italian story; eventually, T–1 decided to replicate Hemingway’s choice
(Schornsteinsegler), arguing that Hemingway ‘imported … his images, and
that he simply saw in the sky the birds that he knew, no matter whether
they were there or not’. In contrast, T–2 concluded that ‘an American
swift … in Padua … this is nonsense, Hemingway knew chimney swifts
from back home’ and therefore decided to use a different bird from the
swift family that lives in Europe (Mauersegler). For T–3 and T–4, the
determining factor was the image (briefly touched upon by only one of
the post-editors, PE–1). T–3 reflected at length on the different images
evoked by the two German terms that exist for the American bird, i.e.
Kaminsegler and Schornsteinsegler (like the English term, they are com-
pounds with the first element denoting different kinds of chimneys that
evoke different images) and then opted for Kaminsegler. T–4 also based
her decision on the visual image, appreciating the ‘beautiful image’ cre-
ated by Mauersegler. T–5 found the term Rauchschwalbe (barn swallow in
English) in a dictionary search and was quickly satisfied, given that ‘a swift
is a bird’; coincidentally, she chose a bird that lives on both continents.
We thus encounter five different birds (two European, two American, and
one European-American) in the five translations, compared to just one (the
American) in the five post-edited versions.
In Example 2, the post-edited target texts are also identical, while the
translations again vary. Here, however, priming occurred on the level of
interpretation. Ambiguity is a typical feature of literary writing, includ-
ing Hemingway’s, notwithstanding his ‘Latin clarity’ (Internationales
Literaturfestival Berlin, 2019), highlighting the translator’s role in inter-
pretation and meaning construction. As Hermans (2007, p. 30) phrased it,
different translations ‘flesh out the interpretive potential of a given text’.
The original sentence Luz sat on the bed is ambiguous in that the verb sat
can be read in two ways, either as the action of sitting down (setzte sich
in German) or as the state of being seated (saß in German). Both inter-
pretations make perfect sense in the context of the story, and indeed we
find both options in the translations (setzte sich in two, saß in three trans-
lations), whereas all five post-edited target texts present the same scene,
in all cases the one suggested by DeepL (setzte sich). PE–1 was the only
post-editor who considered the possibility of an alternative reading but
ultimately found DeepL’s version ‘acceptable’; the verbal data and keylogs
of the other four post-editors do not contain any indication that they were
aware of this interpretative potential or questioned the MT suggestion in
any way. In Group 1, two translators automatically, without any obvious
hesitation, chose the second option, while T–1 and T–2 explicitly reflected
on the ambiguity, T–1 even revising her choice twice before settling on her
final interpretation. In T–4’s audio-recording we can hear a slight hesita-
tion before she typed in the sentence, which may be taken as an indicator
of some state of cognitive uncertainty, though it was obviously not strong
enough to prompt further deliberations.
Example 3 is taken from a later part of the story and serves to illustrate
two more instances of priming, now regarding syntax normalization and
literalism, both indicative of post-editese (italics added):
Original: When they had to say good-bye, in the station at Milan, they
kissed good-bye, but were not finished with the quarrel. (Hemingway,
1925/1930, p. 84)
DeepL: Als sie sich im Bahnhof von Mailand verabschieden mussten,

verabschiedeten sie sich, waren aber mit dem Streit noch nicht fertig.
[English gloss: When they in the station of Milan had to say good-bye,
they said good-bye, but were not yet done with the quarrel.]
DeepL normalized the syntax of the first part of the sentence, shifting the
phrase in the station at Milan to the front of had to say good-bye. In
German, retaining the original’s syntax is possible but results in a slightly
unusual rhythm; four of the five translators opted for it, while four of
the five post-editors retained the normalized German syntax suggested by
64 Waltraud Kolb
DeepL, again, without any hesitation. Of some interest is also the preposi-
tion im (in the). Both in English and German, one could use either in the
station (im Bahnhof) or at the station (am Bahnhof), and in both lan-
guages, the second option leaves more room for interpretation (where
exactly do they say good-bye4: in front of the building, inside the building,
by the train’s door on the platform?). In both languages, the second option
would also be the more common way to describe the scene (we can only
speculate that Hemingway might have opted for in here to avoid another
at in the same phrase). The MT suggestion (im) is a literal translation
and was retained by four post-editors (the exception being PE–5), while
three of the five translators opted for the less literal and more common
(and ambiguous) am (at the). The following excerpt from PE–3’s TAP is
particularly interesting (italics highlight the words of interest and do not
indicate any change in intonation):
PE–3: [reading German MT:] als sie sich am, im Bahnhof von Mailand
verabschiedeten, mussten … verabschiedeten sie sich [laughs], [reading
English original:] they kissed good-bye, [translating:] küssten sie sich
[…] [reading German MT:] im Bahnhof, well, ok … well now we have
one, two, three saying good-bye, one of them completely wrong
We can hear how PE–3 stumbles when she first reads out loud the German
MT version, subconsciously correcting the MT output into the more com-
mon am (at the); she then immediately corrects herself and reads what is
actually on her screen (im); a second later she seems to experience another
moment of uncertainty (‘im Bahnhof, well, ok’), but does not react in any
way to it, probably because the MT error (omission of kissed) monopolizes
her attention. As Krings (2001) has shown, post-editors ‘tend to intervene
correctively in the machine translation subconsciously during reading and
to recognize the correct (and thus familiar) well-formed nature of a text
even where it is not present at all’ (p. 364). And indeed, when reading out
loud the raw MT output or their various interim or final versions, all post-
editors (with the exception of PE–4) subconsciously switched several times
between im (in the) and am (at the).
2.5 Conclusion
Post-edited target texts are synthetic texts that are part human-generated
and part machine-generated, containing both edited and unedited mate-
rial. In the examples discussed above, PE resulted in target texts that
contain distinct traces of priming through MT and are more similar to
4 Hemingway’s spelling.
each other than the translations from scratch. Even though the post-
editors in this study went well beyond securing accuracy and fluency,
their target texts still do not ‘flaunt [their] identity through the difference
with other translative interpretations’ (Hermans, 2007, p. 31) the same
way translations do; put differently, the voices of the post-editors are less
manifest in the target texts than those of the translators (see also Kenny
and Winters, 2020).
On average, PE was found to be faster than HT and involved fewer
keystrokes; however, inter-subject variation regarding speed and work-
ing style was considerable in both groups of participants. Regarding the
underlying cognitive processes associated with the two modalities, dif-
ferent patterns emerged. In line with the post-editors’ role as evaluators
(Guerberof-Arenas & Toral, 2022), their attention was primarily on the
raw MT output, e.g. verifying terminology (chimney swifts) or identifying
and correcting MT errors (omission of kissing); at the same time, their
engagement with the source text and its author’s choices (and on occa-
sion also their final target texts, e.g. regarding imagery) was less extensive
and remained more superficial than in the case of the translators. This
is closely linked to priming effects, which occurred on multiple levels in
the PE modality, be it on the level of terminology (chimney swifts), inter-
pretation (sat), normalization or literalism (in the station). Some of these
priming effects were more subtle than others, but in each case, several of
the post-editors were primed by the same MT suggestion. Together with
the fact that priming occurred on so many levels despite Hemingway’s
rather straightforward language, this clearly points to the great impact
of MT stimuli in a PE task, an impact which can be assumed to be even
greater with stylistically more demanding source texts. While individual
post-edited target texts are not necessarily of lower quality, MT priming
seems to be an intrinsic element of PE processes. Priming effects are here
shown to constrain the post-editor’s agency and lead to more homogene-
ous target texts. It is likely that, in the long run, they will also lead to vari-
ous impoverishments of translated literary language (see also Toral, 2019;
Vanmassenhove et al., 2019).
As the participants in this study had no prior PE training or experience,
further studies will be needed to ascertain whether priming would have
less impact for more experienced post-editors. As far as research tools go,
TAPs provide very rich data for learning more about how HT and PE differ
in terms of translatorial cognition and decision-making; in future studies,
eye-tracking might yield additional insights into patterns of attention
distribution and cognitive rhythms. We can assume that at least in some
sectors of the book market, MT with subsequent human PE will become
more widespread before long (see also Chapter 7 in the present volume).
Therefore, it seems of paramount importance that both literary translators
66 Waltraud Kolb
who take on PE work and publishers who commission it are aware of the
potential implications entailed by the specificities of the cognitive processes
associated with literary PE.
References
Angelone, E. (2010). Uncertainty, uncertainty management and metacognitive
problem solving in the translation task. In G. M. Shreve & E. Angelone (Eds.),
Translation and cognition (pp. 17–40). Benjamins. https://doi.org/10.1075/ata
.xv.03ang
Borg, C. (2022). A literary translation in the making: A process-oriented perspective.
Castilho, S., & Resende, N. (2022a). Post-editese in literary translations.
Information 13(2), 66. https://doi.org/10.3390/info13020066
Castilho, S., & Resende, N. (2022b). MT-pese: Machine translation and post-
editese. Proceedings of the 23rd Annual Conference of the European Association
for Machine Translation (pp. 305–306). https://lt3.ugent.be/media/uploads/
eamt2022/proceedings- eamt2022.pdf
Castilho, S., Resende, N., & Mitkov, R. (2019). What influences the features of
post-editese? A preliminary study. Proceedings of the 2nd Workshop on Human-
Informed Translation and Interpreting Technology (pp. 19–27). https://doi.org
/10.26615/issn.2683-0078.2019_003
Čulo, O., & Nitzke, J. (2016). Patterns of terminological variation in post-editing
and of cognate use in machine translation in contrast to human translation.
Baltic Journal of Modern Computing 4(2), 106–114. https://aclanthology.org
/W16-3401/
Daems, J., De Clercq, O., & Macken, L. (2017). Translationese and post-editese:
How comparable is comparable quality? Linguistica Antverpiensia 16, 89–103.
https://lans-tts.uantwerpen.be/index.php/LANS-TTS/article/view/434
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis. Verbal reports as data.
MIT Press.
European Commission, Directorate-General for Education, Youth, Sport and
Culture. (2022). Translators on the cover: multilingualism & translation:
Report of the Open Method of Coordination (OMC) working group of EU
Member State experts. Publications Office of the European Union. https://data
.europa.eu/doi/10.2766/017
Farrell, M. (2018). Machine translation markers in post-edited machine
translation output. Proceedings of the 40th Conference Translating and the
Computer (pp. 50–59). https://www.asling.org/tc40/wp-content/uploads/
TC40-Proceedings
Guerberof-Arenas, A., & Toral, A. (2020). The impact of post-editing and machine
translation on creativity and reading experience. Translation Spaces 9(2), 255–
282. https://doi.org/10.1075/ts.20035.gue
Guerberof-Arenas, A., & Toral, A. (2022). Creativity in translation: Machine
translation as a constraint for literary texts. Translation Spaces 11(2), 184–212.
https://doi.org/10.1075/ts.21025.gue
Hemingway, E. (1925/1930). A very short story. In E. Hemingway, In our time

(pp. 83–85). Scribner.
Hermans, T. (2007). The conference of the tongues. Routledge. https://www
.academia . edu / 31205772 / Theo _ Hermans _ 2007 _ The _ Conference _ of _ the _
Tongues_Manchester_UK_St_Jerome_Publishing_Pages_190
Internationales Literaturfestival Berlin. (2019). Potentialities of machine
translation. Retrieved December 9, 2022, from https://www.youtube.com/
watch?v=7MTHhjYvXR8
Jakobsen, A. L., & Schou, L. (1999). Translog documentation. In G. Hansen
(Ed.), Probing the process in translation: Methods and results (pp. 151–
186). Samfundslitteratur. https://research.cbs.dk/en/publications/translog
-documentation-version-10
Jones, F. R. (2011). Poetry translating as expert action: Processes, priorities and
networks. Benjamins. https://doi.org/10.1075/btl.93
Kenny, D., & Winters, M. (2020). Machine translation, ethics and the literary
translator’s voice. Translation Spaces 9(1), 123–149. https://doi.org/10.1075/
ts.00024.ken
Kolb, W. (2011). The making of literary translations: Repetition and ambiguity
in a short story by Ernest Hemingway. Across Languages and Cultures 12(2),
259–274. https://akjournals.com/view/journals/084/12/2/article-p259.xml
Kolb, W. (2013). ‘Who are they?’: Decision-making in literary translation. In C.
Way, S. Vandepitte, R. Meylaerts, & M. Bartlomiejczyk (Eds.), Tracks and
treks in translation studies (pp. 207–221). Benjamins. https://doi.org/10.1075
/btl.108.11kol
Kolb, W. (2017). ‘It was on my mind all day’: Literary translators working from
home — Some implications of workplace dynamics. Translation Spaces 6(1),
27–43. https://doi.org/10.1075/ts.6.1.02kol
Kolb, W. (2021). ‘Hemingway’s priorities were just different’: Self-concepts of
literary translators. In K. Kaindl, W. Kolb, & D. Schlager (Eds.), Literary
translator studies (pp. 107–121). Benjamins. https://doi.org/10.1075/btl.156
.05kol
Koponen, M. (2016). Is machine translation post-editing worth the effort? A survey
of research into post-editing and effort. Journal of Specialised Translation 25,
131–148. https://www.jostrans.org/issue25/art_koponen.pdf
Krings, H. P. (2001). Repairing texts: Empirical investigations of machine
translation post-editing processes. Kent State University Press. https://doi.org
/10.7202/008026ar
Lardilleux, A., & Lepage, Y. (2017). CHARCUT: Human-targeted character-based
MT evaluation with loose differences. Proceedings of the 14th International
Workshop on Spoken Language Translation, 146–153. https://aclanthology
.org/2017.iwslt-1.20/
Leijten, M., & Van Waes, L. (2013). Keystroke logging in writing research: Using
Inputlog to analyze writing processes. Written Communication 30(3), 358–392.
https://doi.org/10.1177/0741088313491692
Macken, L., Vanroy, B., Desmet, L., & Teczan, A. (2022). Literary translation as a
three-stage process: Machine translation, post-editing and revision. Proceedings
68 Waltraud Kolb
of the 23rd Annual Conference of the European Association for Machine

Translation (pp. 101–110). https://aclanthology.org/2022.eamt-1.13
Mesa-Lao, B. (2014). Gaze behaviour on source texts: An exploratory study
comparing translation and post-editing. In S. O’Brien, L. Winther Balling,
M. Carl, M. Simard, & L. Specia (Eds.), Post-editing of machine translation:
Processes and applications (pp. 219–245). Cambridge Scholars Publishing.
https://research .cbs .dk /en /publications /gaze -behaviour -on-source -texts -an
-exploratory-study-comparing-tra
Moorkens, J., Toral, A., Castilho, S., & Way, A. (2018). Translators’ perceptions
of literary post-editing using statistical and neural machine translation.
Translation Spaces 7(2), 240–262. https://doi.org/10.1075/ts.18014.moo
O’Brien, S. (2021). Post-editing. In Y. Gambier & L. van Doorslaer (Eds.),
Handbook of translation studies (vol. 5, pp. 177–183). Benjamins. https://doi
.org/10.1075/hts.5.pos3
Toral, A. (2019). Post-editese: An exacerbated translationese. Proceedings of MT
Summit XVII, Volume 1: Research Track, Dublin, Ireland (pp. 273–281).
-6627/
Toral, A., Wieling, M., & Way, A. (2018). Post-editing effort of a novel with
statistical and neural machine translation. Frontiers in Digital Humanities, 5
(Art. 9), 1–11. https://doi.org/10.3389/fdigh.2018.00009
Vanmassenhove, E., Shterionov, D., & Way, A. (2019). Lost in translation: Loss
and decay of linguistic richness in machine translation. Proceedings of MT
Summit XVII, Volume 1: Research Track, Dublin, Ireland (pp. 222–232).
-6622.pdf
Vieira, L. N. (2015). Cognitive effort in post-editing of machine translation:
Evidence from eye movements, subjective ratings, and think-aloud protocols
[Doctoral dissertation]. Newcastle University. https://core.ac.uk/download/pdf
/73983801.pdf
Volkart, L., & Bouillon, P. (2022). Studying post-editese in a professional context:
A pilot study. Proceedings of the 23rd Annual Conference of the European
Association for Machine Translation (pp. 71–79). https://lt3.ugent.be/media/
uploads/eamt2022/proceedings- eamt2022.pdf
3 Mark My Keywords
A Translator-Specific Exploration of Style in
Literary Machine Translation
Marion Winters and Dorothy Kenny
3.1 Introduction
Since the mid-2010s, growing attention has been paid to how machine
translation (MT) might be applied in the translation of literary texts, to
the extent that literary MT is now emerging as a field in its own right. The
interests of researchers working in the area are many and varied (see Kenny
and Winters, forthcoming; and the other chapters in the present volume),
but one important strand focuses on the customisation of MT engines to
improve their performance in translating literary texts. Such customisation
can be individualised: engines can be (partly) trained on texts written by
particular authors (see Chapters 1 and 6, this volume) and/or translated
by particular translators, yielding what we call ‘author-specific personali-
sation’ and ‘translator-specific personalisation’, respectively (Kenny and
Winters, op. cit.). This kind of personalisation has as its implicit or explicit
aim the recreation of a certain ‘style’ in MT, namely that of the source-text
author or the individual translator. It happens upstream, that is, at the
training stage, before any new translation is attempted. It is also possible
to imagine downstream personalisation, however, in the guise of literary
post-editing. In Kenny and Winters (op. cit.) we thus characterise the post-
editing of a machine-translated text by an experienced literary translator
whose brief is to ‘make the text his [i.e. Oeser’s] own’ as a kind of down-
stream translator-specific personalisation (TSP). Whether or not the TSP
in question is successful would depend not on how the outputs were rated
using standard MT evaluation metrics (e.g. Castilho et al., 2018; Way,
2018), but rather on whether the post-edited version could be shown to
reflect the translator’s known style. In this chapter, we continue on this
track, presenting the results of an empirical investigation into literary
translator Hans-Christian Oeser’s style when he works in post-editing as
DOI: 10.4324/9781003357391-5
70 Marion Winters and Dorothy Kenny
opposed to ‘conventional’ translation mode.1 In what follows, we first

describe how we approach style in this study and share what we already
know about Oeser’s style as a translator. We then describe the specific
method we use to analyse translator style in Oeser’s post-edited text,
which relies on keyword analysis, before outlining the rest of our research
design. Finally, we present our results and discuss our findings in light of
what they can tell us about downstream TSP.
3.2 Literary Style and Translation

Interest in literary style goes back centuries, but scholars have not always
proceeded on the basis of a shared understanding of the term (Herrmann
et al., 2015, p. 25). Even within the bounds of Translation Studies
treatments of style vary (see, especially, Saldanha, 2014), but there is a
general consensus that one can study the style of source texts, target texts
or translators. Saldanha’s (2011) definition of translator style has been
particularly influential, both in our own work and elsewhere (e.g. Kenny
and Winters, 2020; Youdale, 2020). For reasons outlined in Kenny and
Winters (forthcoming), however, we prefer to fall back on the broader
definition of style of Herrmann et al. (2015, p. 44) as:
a property of texts constituted by an ensemble of formal features which

can be observed quantitatively or qualitatively. (italics in original)
and then to hypothesise causes for the observed formal features of texts, in
the form of particular translators, post-editors, and so on.
The particular formal features that can be studied and the ways in which
they can be processed run into the hundreds (Herrmann et al., 2015, p. 45).
Previous corpus-based and corpus-driven studies in translator style (e.g.
Baker, 2000; Saldanha, 2011; Youdale, 2020) have started, for example,
by focusing on basic statistics like lexical density, average sentence length,
(standardised) type–token ratios and other measures of lexical variety. These
studies usually branch into richer qualitative analyses on the basis of their
initial quantitative findings. Similar metrics have been used in studies that
1 We are using ‘conventional’ here to designate translation completed without the use of MT
for lack of a better term. Alternative terms used in the literature create as many problems
as they solve: ‘human translation’, for example, is often used to contrast with ‘machine
translation’ but post-editing is very much a human activity too, a fact that reduces the
discriminating capacity of the epithet ‘human’ in this instance. Likewise, the term ‘from
scratch’ seems inadequate as it appears to present translation completed without the use of
MT as a kind of ex nihilo activity, which is clearly not the case. That said, ‘from scratch’ is
so commonly used to designate translation completed without a machine-translated first
draft that it is difficult to displace.
Mark My Keywords 71
compare the style of machine-translated and human-translated texts (e.g.

Lee, 2021) and in investigations of ‘post-editese’ (Toral, 2019), although
neither Lee nor Toral focuses on any particular translator or post-editor.
Somewhat more sophisticated quantitative analyses use Burrows’ (2002)
Delta to calculate distances between texts. This approach is favoured by
scholars working in the wider Digital Humanities (e.g. Rybicki, 2012)
where the predominant concern is with author/translator attribution. In
related work that uses the most frequent 1-, 2- and 3-grams (1-, 2- and
3-word sequences, not necessarily phrases per se) as the features of interest,
Lee (2021) applies standard machine learning and statistical techniques
(support vector machines and principal component analysis respectively)
to differentiate human and machine translations in what is essentially a
text classification task; as already indicated, Lee’s focus is not on the style
of any particular translator, however. For him, ‘human translators’ remain
an undifferentiated mass, whereas ‘machine translators’ (i.e. MT systems)
are identified by name.
In the current study, we investigate Oeser’s style using lexical features—
in this case word forms—that stand out because of their unusually high
frequency in a text he has post-edited compared to the MT output that
he starts with. Their unusual frequency in Oeser’s post-edited text makes
them keywords (see below) in that text. We go on to compare Oeser’s post-
edited text with a corpus of his other recent translations, and with a corpus
of original German literary prose, in a bid to see whether Oeser’s post-edits
are consistent with his wider work and with German fiction in general. We
do this using a second, more focused, iteration of keyword analysis.
3.3 Oeser’s Style as a Translator and Post-Editor

Hans-Christian Oeser is an internationally acclaimed literary translator
with more than 40 years’ experience and over 220 titles to his name as
translator, editor or author.2 In previous work, Winters (2007, 2009) used
corpus-driven and comparative methods to study his style as a translator.
Winters (2015) uses interview data to triangulate findings from these
studies. We have thus built up a rich picture of Oeser’s observed and self-
reported style, some prominent aspects of which are that he:
• favours subject-verb inversion

• is committed to preserving lexical richness in translation
• very consciously uses higher register when appropriate
• attempts to replicate features of natural spoken language
2 See https://hanschristianoeser.wixsite.com/hcoeser for further details.

In Kenny and Winters (2020), we use an experimental design to see what

happens to Oeser’s style when he is called upon to post-edit rather than
translate conventionally, finding that, overall, it is somewhat diminished.
The current study improves on this previous work in particular by drawing
on a real translation brief.
3.4 Keywords as a Method in Investigations of Style

Herrmann et al. (2015, p. 45) point to the vast number and heterogeneity
of potential stylistic features and measures encountered in the literature.
Nevertheless, they contend that ‘most style markers have so far been
relatively simple in nature’. They include:
frequencies and frequency distributions of characters, words, lem-

mata, word classes or syntactical structures, taken by themselves or
in sequences (n–grams); and the length, and distribution of lengths, of
words, sentences, paragraphs or other units.
(ibid.)
Higher-order stylistic features are derived from these basic style markers by
relating selected markers to each other and/or through the application of
various statistical techniques and tests. Among the ‘well understood’ (ibid.)
methods used to generate such higher-order features, Herrmann et al. (ibid.)
list keyness measures. In short, an item is considered ‘key’ in a given text or
corpus if it occurs with unusual frequency compared with its frequency in
another text or (often larger) corpus. The second (larger) corpus is usually
called the reference corpus, although Scott (2022) uses the term comparison
corpus, a usage we follow in this chapter. Although keyness can be attributed
to any of the items listed above (lemmata, word classes, etc.) it is frequently
word forms that are studied in corpus-based translation studies. In the
paradigm case (see Rayson, 2019), a word form (or ‘type’) is said to be a
keyword in a given text, if it occurs with greater than expected frequency in
that text given its frequency in the comparison corpus. Such a keyword is
described as a positive keyword. A negative keyword, in contrast, is one that
occurs with less than expected frequency in the text in question.
The computation of keyness usually involves comparing the relative
frequencies of word forms in the focus text and the comparison corpus,
and then conducting a test to ascertain whether any difference between
these relative frequencies is statistically significant. The score produced by
this test can then be used to rank word forms in terms of their keyness in
the text in question.
Mark My Keywords 73
There has been much debate over which statistical significance test is
most suitable for the identification and ranking of keywords, as well as
general criticism over assumptions made by many of the tests in question
(see Rayson, 2019). The log likelihood test has been favoured for some
time (ibid.), although it is not without its detractors (e.g. Gabrielatos,
2018). Jeaco (2020, p. 149), however, holds that log likelihood-based
keyword calculations ‘can be used effectively for a range of different
kinds of research, but often work best with texts and moderately large
collections of texts rather than with very large corpora at the entire
corpus level’. He adds (ibid.) that log likelihood can be used effectively in
combination with related measures such as Bayesian Information Criterion
(BIC). The corpus-processing software WordSmith Tools now integrates
BIC alongside a variety of keyness measures including log likelihood, the
settings for which are often user-adjustable.3
Keyword analysis has also come under criticism for its prioritisation
of difference over similarity in corpus studies (see e.g. Taylor, 2018). But
even the techniques most associated with the privileging of difference
can be turned to the analysis of similarity (ibid., p. 21) and it is possible,
for example, to investigate whether keywords generated for a given text
remain key in other texts by the same author or translator, given a different
comparison corpus. This is the approach taken in this study.
All told, keyword analysis offers a number of advantages in corpus
stylistic studies. Mastropierro (2018, p. 66), for example, argues that
using keywords means that the analyst works with a controlled number of
automatically generated items, whose frequency is statistically significant,
which makes for ‘an efficient way to begin a study’ and helps to minimise
researcher bias. Keyword lists typically contain content and function words,
both of which can characterise an author’s, translator’s or character’s style
(Culpeper, 2009) and thus are of interest to stylisticians. This fact can
differentiate keyword analysis from ‘most frequent word’ analyses, which
typically revolve around function words. Keywords also provide a useful
exploratory bridge between more quantitative analyses of textual features
and more qualitative, interpretative analyses, as is customary in corpus
stylistics (Herrmann, 2017). And, as Mastropierro (2018, p. 67) reminds
us, the generation of a keyword list is not an end in itself; rather, it can be
the starting point for further quantitative and qualitative examination of
what, upon careful analysis, is likely to turn out to be important meaningful
features of the text in question.
3 www.lexically.net/wordsmith/.
3.5 Research Design
3.5.1 Data
In this study, we use as primary data two German versions of Christopher

Isherwood’s (1954) novel The World in the Evening. The first is a machine
translation initiated by Hans-Christian Oeser using the free version of
DeepL in 2019.4 The second is Oeser’s post-edited translation of the
DeepL output, subsequently published as Die Welt am Abend (Isherwood,
2019). In what follows we label these texts DeepL MT and Oeser PE,
respectively. We received both texts directly from Oeser.
If using DeepL was ‘a cakewalk’ (Oeser, 2020, p. 21), then post-editing its
output to create a publishable translation that would meet Oeser’s normal
standards turned out to be less straightforward. Although he describes
the experience as ‘somewhat less time-consuming’ than translating ‘from
scratch’, the process entailed ‘painstaking retranslation’ given that ‘there
was hardly a sentence that did not have to be thoroughly revised and
rebuilt’ (ibid., p. 22). The post-editing was completed in Oeser’s normal
working environment using Microsoft Word. Given that the maintenance
of ecological validity was particularly important to us and to Oeser, no
observation techniques were used to track Oeser’s progress, and no attempt
was made to time Oeser or to gather any kind of user-activity data.
Table 3.1 gives a basic quantitative overview of the texts,5 presenting
token counts, (unlemmatised) type counts and type–token ratios—unstand-
ardised (TTR) and standardised (STTR) using a base of 1,000—all as com-
puted by WordSmith version 8.0 using the software’s default settings.
Table 3.1 shows that Oeser decreases the token count and simultane-
ously increases lexical variety, as evidenced by higher type–token ratios in
Oeser PE than DeepL MT. This superficial analysis tells us nothing about
the particular edits Oeser makes, however. Our keywords analysis below
will shed some light on this issue.
Table 3.1 Basic for statistics for DeepL MT and Oeser PE.
tokens types ttr sttr

DeepL MT 104,892 10,603 10.11 45.84
Oeser PE 103,511 12,210 11.80 47.55
4 www.deepl.com/translator (last accessed 30 September 2022).

5 The corresponding statistics for the source text are: 104,096 tokens; 8,123 types; TTR
7.80; STTR 43.35.
Mark My Keywords 75
We are interested not just in how Oeser’s post-edited version differs

from the MT version, but also in whether the interventions Oeser makes
as a post-editor are consistent with his translatorial style. We thus compare
Oeser PE with a purpose-built corpus, Oeser 12, containing 12 novels/
novellas translated by Oeser as the sole translator over a roughly contem-
poraneous period (2016 to 2021).6
A third comparison, this time with a corpus called Original German
Literature, sheds further light on Oeser’s style, indicating whether it is
distinctive from or consistent with that of other German-language authors.
Original German Literature runs to 3,596,676 tokens. It contains 57 novels
and novellas (excluding crime fiction) extracted from the 53 billion-word7
German Reference Corpus of the Institut für Deutsche Sprache (IDS).
These 57 works were published between 2000 and 2012, the most recent
year covered by the IDS.8
Although our analysis is primarily target-oriented, our qualitative
analysis sometimes requires us to look back at the source text to seek extra
contextual information about what prompted particular translations. For
this purpose, we use a small parallel corpus consisting of the source text
(Isherwood ST) and two target texts (DeepL MT and Oeser PE) aligned at
paragraph level and accessed using Tetrapla (Woolls, 2021).9
3.5.2 Procedure
We start our analysis by generating a keyword list for Oeser PE using the
keywords function in WordSmith Tools 8.0 and taking DeepL MT as our
comparison corpus.10 In general, the closer the comparison corpus is to the
text under study in terms of genre and other extratextual factors the better
(Culpeper, 2009). In our case, the comparison corpus is extremely close to
the study text, differing only in the translation condition (post-edited MT
vs raw MT). This reduces the chances that the keyword analysis will high-
light words that indicate what the study text is ‘about’, which might serve
as a distractor in a study of style (Scott & Tribble, 2006). The output is a
6 Oeser 12 contains 699,315 tokens and 52,427 types and has a standardized TTR of 50.47.
7 Correct as of 08 March 2022.
8 The IDS allows users to build a customised sub-corpus from the German Reference Cor-
pus. However, there is no function to create frequency lists or other statistics. Counts for
selected types (see Section 3.6.4) were thus done by looking them up individually.
9 We would like to thank David Woolls for developing Tetrapla and optimizing it for Ger-
man.
10 The relevant settings are: maximum wanted = 500; minimum frequency of occurrence of
candidate keywords in the study text = 3; minimum BIC score = 2.5; minimum log ratio
= 0; p value = 0.1. The p value is set high as it can effectively be ignored if BIC values are
used (Scott, 2022).
Table 3.2 Positive keywords in Oeser PE compared to DeepL MT.
N Key word Freq. in % in Freq. in % in BIC Log_L

Oeser Oeser DeepL DeepL MT
PE PE MT
1 ACH 144 0.14 0 0.00 189.20 201.45
2 DOCH 275 0.27 62 0.06 135.88 148.13
3 GAR 81 0.08 4 0.00 74.30 86.55
4 S 70 0.07 4 0.00 60.05 72.30
5 SIE 2,370 2.29 1,875 1.79 52.01 64.26
6 ELIZABETHS 49 0.05 2 0.00 42.17 54.42
7 VERMUTLICH 36 0.03 0 0.00 38.11 50.36
8 BESTIMMT 31 0.03 0 0.00 31.12 43.37
9 SCHON 183 0.18 83 0.08 27.56 39.80
10 FALLS 37 0.04 2 0.00 26.48 38.73
11 EBEN 33 0.03 1 0.00 26.27 38.52
12 ELIZABETH 372 0.36 226 0.22 25.62 37.87
13 FURCHTBAR 36 0.03 3 0.00 21.08 33.33
14 GEWISS 20 0.02 0 0.00 15.73 27.98
15 WESHALB 19 0.02 0 0.00 14.33 26.58
16 HAB 24 0.02 1 0.00 14.30 26.55
17 O 30 0.03 3 0.00 13.74 25.98
18 MERKTE 18 0.02 0 0.00 12.93 25.18
19 JA 230 0.22 138 0.13 12.17 24.42
20 GERN 17 0.02 0 0.00 11.53 23.78
21 STETS 17 0.02 0 0.00 11.53 23.78
22 JENEM 17 0.02 0 0.00 11.53 23.78
23 SCHWARZENSEE 16 0.02 0 0.00 10.13 22.38
list of candidate keywords ranked in descending order of their log likeli-

hood and BIC score (see Table 3.2).
Because we are also interested in whether Oeser’s post-edits reaffirm
his style as observed elsewhere, and whether these features of his style
differentiate him from other writers in German, we go on to compare the
relative frequencies of keywords identified in Oeser PE with their relative
frequencies in our two reference corpora: Oeser 12 and Original German
Literature. We test whether any differences observed are statistically
significant, again using log likelihood and BIC scores, this time calculated
using Rayson’s log-likelihood and effect size calculator.11 This effectively
means that we test the keyness in other data sets of a handful of positive
keywords identified in Oeser PE.
11 Available from https://ucrel.lancs.ac.uk/llwizard.html.

Mark My Keywords 77
3.6 Results and Discussion

Table 3.2 shows all 23 positive keywords in Oeser PE generated using
DeepL MT as a comparison corpus.12 Space constraints prevent us from
discussing all of them, so in the following, we attempt to account primarily
for groups of related keywords.13
3.6.1 Proper Nouns and Inconsistent MT
All keywords proposed in Table 3.2 tell us something interesting about

changes that Oeser has made to the machine-translated text. Even proper
nouns (ranked 6, 12 and 23) throw up surprises: although it is common to
find proper nouns in keyword lists, this was not expected to happen in the
current case, given the extremely close relationship between the study text
and the comparison corpus and thus apparently complete overlap of their
‘aboutness’. The explanation for this finding is trivial, however. DeepL
is inconsistent in the spelling (s vs z) of its translation of Elizabeth and
Elizabeth’s. Such inconsistency in lexical translation is a well-known prob-
lem even in state-of-the-art neural MT systems. Oeser fixes the problem
by standardising to Elizabeth/Elizabeths. What appears in Table 3.2 as
a keyword in his post-edited text is actually the trace of error correction
rather than a reflection of his style. The other proper noun (Schwarzensee,
ranked 23) appears because Oeser changes the spelling for the lake known
as the Schwarzsee in both the source text and DeepL MT.
3.6.2 Fictional Dialogue
More interesting are the seemingly innocuous words s (ranked 4; a

contracted form of es ‘it’) and hab (ranked 16; a contracted form of [ich]
12 125 negative keywords were generated in Oeser PE using the same settings. Due to space
constraints, we will not elaborate on them here except to the extent that they shed light
on the positive keywords we discuss.
13 Log likelihood (LL) values can be interpreted as follows, according to Rayson (See fn. 11):
LL of 3.8 or higher is significant at the level of p < 0.05
LL of 6.6 or higher is significant at p < 0.01
95th percentile; 5% level; p < 0.05; critical value = 3.84
99th percentile; 1% level; p < 0.01; critical value = 6.63
99.9th percentile; 0.1% level; p < 0.001; critical value = 10.83
99.99th percentile; 0.01% level; p < 0.0001; critical value = 15.13
BIC scores can be interpreted as follows, according to Gabrielatos (2018):
below 0: not trustworthy (or evidence in favour of H0 according to Rayson (ibid.)
0–2: not worth more than a bare mention
2–6: positive evidence against H0
6–10: strong evidence against H0
>10: very strong evidence against H0
habe ‘[I] have’). They are used in depictions of direct speech by Oeser
but not by DeepL. The elevated frequency of these forms in Oeser PE
is consistent with what is already known about Oeser’s skill in writing
convincing dialogue.
Oeser’s frequent use of interjections and (potential) modal particles
such as ach (ranked 1), doch (ranked 2), gar (ranked 3), schon (ranked
9), eben (ranked 11) and ja (ranked 19) may also be interpretable in this
light, although it should be noted that instances of ach and o (ranked 17)
tend to be straightforward replacements for the untranslated particle Oh
in DeepL MT, and that the other forms mentioned here are generally
polysemous, so each instance needs to be inspected individually to come
to sound conclusions. Even the overrepresentation of the polysemous
pronoun sie/Sie (ranked 5), one use of which is as a formal translation of
‘you’, is at least partly attributable to its use in Oeser’s fictional dialogue.
Nearly half of the keywords identified in Table 3.2 may thus be linked
to Oeser’s particular way of handling fictional dialogue. Space restrictions
prevent us from analysing these data further here. For now, we simply note
the importance of keyword analysis in leading us towards higher-order
features of texts (such as the treatment of dialogue) that differentiate the
post-editor’s work from that of the machine.
3.6.3 Lexical Preferences and Marker Words in Oeser PE
Of the other keywords in Table 3.2, vermutlich (ranked 7), bestimmt

(ranked 8) and gewiss (ranked 14) form a group: all three function primar-
ily as epistemic modal adjuncts and thus indicate the speaker’s assessment
of the truth of a proposition, for example, whether it is possible, probable
or certain. They translate roughly as ‘probably’ (in the case of vermutlich)
and ‘certainly’ (in the case of bestimmt and gewiss). The combined fre-
quency of these three forms in Oeser PE is 87, while there are no occur-
rences at all in DeepL MT. All three represent ‘marker words’ for Oeser,
that is words that reflect particular likes or dislikes of an author, compared
to a ‘competing’ author (Kenny, 1982, p. 8), in this case DeepL. We ana-
lyse each of these three keywords in more detail below.
3.6.3.1 Vermutlich
A parallel concordance for vermutlich in Oeser PE alongside the

corresponding segments in DeepL MT and Isherwood ST reveals that
DeepL had output ich nehme an for ‘I suppose’ in 17 instances and
wahrscheinlich for ‘probably’ in 16 instances. In two cases where Oeser
used vermutlich, DeepL had output ich denke. In one case vermutlich
replaced ich schätze (see Table 3.3).
Mark My Keywords 79
Table 3.3 Vermutlich in Oeser PE and corresponding items in DeepL MT and

Isherwood ST.
Oeser PE DeepL MT Isherwood ST

Vermutlich (36) ich nehme an (17) I suppose (17)
wahrscheinlich (16) probably (16)
ich denke (2) I expect (1)
I think (1)
ich schätze (1) I guess (1)
In no case does Oeser’s edit affect the meaning or epistemic stance of the
speaker. The changes from the verbal ich nehme an to the adverbial ver-
mutlich do, however, influence the sentence structure, which changes from
a hypotactic structure to a simple main clause in 12 instances (Example 1)
or from a main clause to an incomplete sentence in five instances (Example
2).14 The former allows inversion, which Oeser is known to like (see
Section 3.3). The latter, it could be argued, yields a better approximation
of spoken language.
Example 1
(1a) Isherwood ST I suppose I still regarded marriage as a kind of
game.
(1b) DeepL MT Ich nehme an, ich betrachtete die Ehe immer noch
als eine Art Spiel.
(1c) Oeser PE Vermutlich betrachtete ich die Ehe noch immer als
eine Art Spiel.
Example 2
(2a) Isherwood ST ‘Yes. I suppose so.’
(2b) DeepL MT „Ja. Ich nehme es an.”
(2c) Oeser PE „Ja. Vermutlich.”
Likewise, Oeser’s choice of vermutlich to replace the synonymous wahr-

scheinlich has little impact on meaning or register (Example 3).
14 Note that particular examples are chosen as they attest the use of the word form in ques-
tion in a relatively short sentence with few ‘distractors’. By their very nature they may
underrepresent the amount of editing that Oeser does across the text as a whole.
Example 3
(3a) Isherwood ST Yes, the Jane-situation still existed, and would
continue to exist, probably, for a long time.
(3b) DeepL MT Ja, die Jane-Situation existierte noch und würde
wahrscheinlich noch lange bestehen bleiben.
(3c) Oeser PE Ja, die Jane-Situation existierte noch und würde ver-
mutlich noch lange existieren.
Indeed, Oeser could have maintained all instances of wahrscheinlich in

his post-edited version, but he appears to actively dislike the word.15 Not
only does he change it to vermutlich in the 16 instances already noted,
but on no occasion does he introduce wahrscheinlich (e.g. as a possible
replacement for ich nehme an in the DeepL output). Oeser thus reduces the
overall frequency of wahrscheinlich from 61 in the MT output to 42 in his
post-edited text. And when he does use wahrscheinlich, it is always already
present in the MT output.
3.6.3.2 Bestimmt
Table 3.4 presents findings for bestimmt. Out of 31 instances, 28 have an

epistemic function. The other three were used non-epistemically to mean
‘firm’ or ‘firmly’.
Where Oeser uses bestimmt it is frequently to replace sicher (‘sure’) (12
instances) or sicherlich (‘certainly’) (seven instances). Edits from sicher to
bestimmt, it could be argued, involve a slight decrease in assertiveness,
Table 3.4 Bestimmt in Oeser PE and corresponding items in DeepL MT and

Isherwood ST.

bestimmt (28) sicher (12) sure (9), surely (2), certainly (1)
sicherlich (7) certainly (7)
müssen (4) must (3), bound to (1)
definitiv (1) definitely
wahrscheinlich (1) probably
gar (1) certainly
nicht wahr (1) didn’t he
bin ich gefesselt (1) I’ll be bound
15 This appears to be a conscious preference: when asked in an interview (conducted on 22

April 2022) whether he liked the word wahrscheinlich, Oeser immediately replied: ‘Nein,
ich sag meistens vermutlich’. (‘No, I mostly say vermutlich’.)
Mark My Keywords 81
and some edits (five instances) may have been made in an effort to create
natural-sounding dialogue (Example 4).
Example 4
(4a) Isherwood ST ‘I’m sure you look cute in it.’
(4b) DeepL MT „Ich bin sicher, du siehst darin süß aus.”
(4c) Oeser PE „Du siehst bestimmt niedlich darin aus.”
The instances where sicherlich is changed to bestimmt (with or without

inversion), on the other hand, seem to be motivated exclusively by Oeser’s
preference (see Example 5, which also exemplifies inversion).
Example 5
(5a) Isherwood ST I would miss her, certainly.
(5b) DeepL MT Ich würde sie sicherlich vermissen.
(5c) Oeser PE Bestimmt würde ich sie vermissen.
3.6.3.4 Gewiss
Table 3.5 shows findings for gewiss.

Of the 20 instances of gewiss in Oeser PE, 14 replace sicherlich and
five replace sicher (see Examples 6 and 7). These edits may result in a
slight elevation of register (Oeser, personal communication 22/04/2022),
although sources like the Duden dictionary suggest that sicher/sicherlich
and gewiss are synonyms.
Example 6
(6a) Isherwood ST Certainly not because I imagine you’ll disap-
prove of him.
(6b) DeepL MT Sicherlich nicht, weil ich mir vorstelle, dass du ihn
missbilligen wirst.
(6c) Oeser PE Gewiss nicht, weil ich glaube, dass du ihn missbilligen
wirst.
Table 3.5 Gewiss in Oeser PE and corresponding items in DeepL MT and

Isherwood ST.

gewiss (20) sicherlich (14) certainly (13), surely (1)
sicher (5) certainly (2), sure (3)
gar (1) certainly (1)
Example 7
(7a) Isherwood ST ‘Sure, I understand all about that, Bob.
(7b) DeepL MT Sicher, ich verstehe das alles, Bob.
(7c) Oeser PE Gewiss, das alles verstehe ich, Bob.
The flip side of Oeser’s preference for gewiss seems to be a dislike of the
synonymous sicher and sicherlich. He reduces their frequency considerably,
by a third in the case of sicher (from 168 to 106 instances) and by three-
quarters in the case of sicherlich (from 46 to ten instances). Unsurprisingly,
both sicher and sicherlich thus appear as negative keywords for Oeser PE
when DeepL MT is the comparison corpus.16
3.6.3.5 Weshalb
While the above three keywords can be grouped on semantic and prag-
matic grounds, weshalb stands alone as the only interrogative form in the
keyword list in Table 3.2. It translates as ‘why’ and is used by Oeser on
all occasions as an interrogative or relative adverb to replace the more
common synonymous form warum in the DeepL output. It is of par-
ticular interest to us, as its use has previously been identified as one of
Oeser’s ‘quirks’ (Kenny & Winters, 2020, p. 143). Although Oeser regards
weshalb as being of higher register than warum (personal communication,
22/04/2022), the decision to change warum to weshalb is based entirely on
his personal preference.
3.6.4 Oeser PE Keywords in the Reference Corpora
The analysis so far has concentrated on what keywords in Oeser PE

(using DeepL MT as a comparison corpus) tell us about the particu-
lar changes Oeser has made to the machine-translated text, and we
have referred obliquely to how these changes are manifestations of
Oeser’s style as a translator. More compelling direct evidence comes
from comparisons between the frequencies of these keywords in Oeser
PE and Oeser 12, and Oeser PE and German Original Literature (see
Section 3.5.1).
16 Sicher: rank = 132, frequency in Oeser PE = 106, % in Oeser PE = 0.10, frequency in

DeepL MT = 168, % in DeepL MT = 0.16, BIC = 1.13, LL = -13.38, p = 0.0002541700.
Sicherlich: rank = 142, frequency in Oeser PE=10, % in Oeser PE = 0.01, frequency in
DeepL MT = 46, % in DeepL MT = 0.04, BIC=12.38, LL = –24.63, p = 0.0000006923.
Mark My Keywords 83
Table 3.6 gives the absolute and relative frequencies for the four key-
words addressed above in the texts/corpora in question, sorted in descend-
ing order of absolute frequency in Oeser PE.17
As the bolding in Table 3.6 highlights, vermutlich, bestimmt and gewiss
are all relatively more frequent in Oeser PE than in his wider work (Oeser
12). Only weshalb occurs relatively less frequently in Oeser PE than it
does in Oeser 12. All four forms are relatively more common in Oeser PE
than in German Original Literature. The comparison between Oeser 12
and German Original Literature suggests that Oeser generally uses ver-
mutlich and weshalb more than other writers in the target language while
he is less likely than others to use bestimmt and gewiss. In his post-editing
work then, he appears to be asserting his attested lexical style in the use
of vermutlich and weshalb. He uses bestimmt and gewiss, however, not
because they are characteristic of his style, but rather to avoid using sicher
and sicherlich, which he generally disprefers. That said, he ‘overuses’ them
in his post-edited work most likely because of the influence of the MT
(although the source text could also be exerting an influence here) (see
Table 3.7).
As indicated above, differences in frequencies can be tested for statistical
significance using log likelihood, which compares observed and expected
Table 3.6 Frequency comparison of keywords in Oeser PE.
keyword Oeser per DeepL per Oeser per German per

PE 100,000 MT 100,000 12 100,000 Original 100,000
Lit.
vermutlich 36 34.62 0 0.00 127 18.12 269 7.48

bestimmt 31 29.82 0 0.00 118 16.84 654 18.18
gewiss 20 19.24 0 0.00 34 4.85 281 7.81
weshalb 19 18.27 0 0.00 168 23.97 257 7.15
Table 3.7 Lexical items Oeser generally disprefers (sorted by freq. in Oeser PE).
keyword Oeser per DeepL per Oeser per German per

PE 100,000 MT 100,000 12 100,000 Original Lit. 100,000
sicher 106 101.95 168 160.05 197 28.11 1,456 40.48

wahrscheinlich 42 40.39 61 58.11 64 9.13 706 19.63
sicherlich 10 9.62 46 43.82 6 0.86 133 3.70
17 As keyness scores—and hence ranking—depend crucially on the comparison corpus in

use, it is less useful to compare ranked keywords given changing comparison corpora. For
this reason, we present relative frequencies in the first instance.
Table 3.8 Keyness of selected items Oeser PE vs Oeser 12.
observed expected over/under

frequencies frequencies -use
Oeser Oeser Oeser Oeser LogLikelihood BIC
PE 12 PE 12
1. vermutlich 36 127 21.02 141.98 + 10.42 –3.17

2. bestimmt 31 118 19.21 129.79 + 7.19 –6.40
3. gewiss 20 34 6.96 47.04 + 20.14 6.54
4. weshalb 19 168 24.11 162.89 – 1.33 –12.27
Table 3.9 Keyness of selected items Oeser 12 vs German Original Literature.
observed expected over/under

frequencies frequencies -use
Oeser 12 German Oeser 12 German Log BIC
Original Original Likelihood
Literature Literature
1. vermutlich 127 269 64.46 331.54 + 59.78 44.51

2. bestimmt 118 654 125.67 646.33 – 0.57 –14.70
3. gewiss 34 281 51.28 263.72 – 7.72 –7.55
4. weshalb 168 257 69.18 355.82 + 130.88 115.60
frequencies. Table 3.8, based on Rayson’s log-likelihood and effect size

calculator, suggests that vermutlich, bestimmt and gewiss are all signifi-
cantly overused in Oeser PE compared to his wider work in Oeser 12,
although negative BIC scores for the former two suggest that this result
is ‘untrustworthy’ (Gabrielatos, 2018). The evidence for the underuse of
weshalb also appears to be untrustworthy.
Comparisons between Oeser’s wider work and other texts in German
might be more fruitful, however, as the influence of individual texts is
mitigated. Table 3.9 thus indicates that there is strong evidence that ver-
mutlich and weshalb are overused by Oeser in general, which supports the
interpretation of their use in Oeser PE as a manifestation of Oeser’s style.
Meanwhile, evidence to claim that Oeser underuses bestimmt and gewiss
is far weaker.
3.7 Conclusions
This chapter has explored the use of keywords as a way of eliciting data
for the analysis of a post-editing translator’s style. The generation of an
unlemmatised keyword list proved to be an efficient way of eliciting unbi-
ased data for further examination, leading us to at least one higher-order
Mark My Keywords 85
feature that merits further attention, namely Oeser’s treatment of fic-

tional monologue and dialogue. Our comparative approach also allowed
us to see keywords in Oeser’s post-edited text not just as evidence of
systematic editing, but also as indices of his lexical style as attested in
a corpus of his translation work. Two words, vermutlich and weshalb,
turned out to be extremely strong marker words for Oeser, and other
keywords remain to be investigated in full. Although not anticipated at
the outset, negative keywords also turned out to be of interest, given their
ability to indicate the post-editor’s dislikes. That disliked items remained
to the extent they did in the post-edited version of the novel serves as a
reminder of the strong priming influence of the machine-translated text.
Also interesting, from the point of view of studies of style at least, is the
fact that the keywords technique tends to draw the analyst’s attention
not to instances of error correction, which is the focus of much post-edit-
ing research, but to the ways in which the translator/post-editor asserts
his style in the target text by making what are known in the post-editing
literature as ‘preferential’ changes (see O’Brien, 2022, p. 118). Finally,
this research differs from many existing inquiries into post-editing in that
it is not concerned with productivity, and it focuses on a single named
translator/post-editor, thanks to the availability of—to our knowledge—
a unique data set.
References
Baker, M. (2000). Towards a methodology for investigating the style of a literary
translator. Target 12(2), 241–266. https://benjamins.com/online/target/articles
/target.12.2.04bak
Burrows, J.F. (2002). Delta: A measure of stylistic difference and a guide to likely
authorship. Literary and Linguistic Computing 17, 267–287. https://doi.org/10
.1093/llc/17.3.267
Castilho, S., Doherty, S., Gaspari, F. & Moorkens, J. (2018). Approaches to
human and machine translation quality assessment. In J. Moorkens, S. Castilho,
F. Gaspari & S. Doherty (Eds), Translation quality assessment: From principles
to practice (pp. 9–38). Springer. https://www.researchgate.net/publication
/326386260 _ Approaches _ to _ Human _ and _ Machine _ Translation _ Quality
_Assessment_From_Principles_to_Practice
Culpeper, J. (2009). Keyness: Words, parts-of-speech and semantic categories in
the character-talk of Shakespeare’s Romeo and Juliet. International Journal
of Corpus Linguistics 14(1), 29–59. https://www.researchgate.net/publication
/233493422_Keyness_Words_parts-of-speech_and_semantic_categories_in_the
_character-talk_of_Shakespeare's_Romeo_and_Juliet
Gabrielatos, C. (2018). Keyness analysis: Nature, metrics and techniques. In
C. Taylor & A. Marchi (Eds.), Corpus approaches to discourse: A critical
review (pp. 225–258). Routledge. https://www.researchgate.net/publication
/319208347_Keyness_analysis_Nature_metrics_and_techniques
Herrmann, J.B. (2017). In a test bed with Kafka: Introducing a mixed-method

approach to digital stylistics. Digital Humanities Quarterly 11(4). http://www
.digitalhumanities.org/dhq/vol/11/4/000341/000341.html#
Herrmann, J.B., van Dalen-Oskam, K. & Schöch, C. (2015). Revisiting style, a key
concept in literary studies. Journal of Literary Theory 9(1), 25–52. http://www
.jltonline.de/index.php/articles/article/view/757/1776
Isherwood, C. (1954). The World in the Evening. Vintage.
Isherwood, C. (2019). Die Welt am Abend (Trans. H.-C. Oeser). Hoffmann und
Campe.
Jeaco, S. (2020). Key words when text forms the unit of study. Sizing up the effects
of different measures. International Journal of Corpus Linguistics 25(2), 125–
154. https://doi.org/10.1075/ijcl.18053.jea
Kenny, A. (1982). The computation of style. Pergamon Press. https://www
.sciencedirect.com/book/9780080242811/the-computation-of-style
Kenny, D. & Winters, M. (2020). Machine translation, ethics and the literary
translator’s voice. Translation Spaces 9(1), 123–149. https://doi.org/10.1075/
ts.00024.ken
Kenny, D. & Winters, M. (2024). Customization, personalization and style in
literary machine translation. In M. Winters, S. Deane-Cox & U. Böser (Eds.),
Technological change in translation and interpreting. Bloomsbury. (In press).
Lee, C. (2021). How do machine translators measure up to human literary
translators in stylometric tests? Digital Scholarship in the Humanities 37(3),
813–829. https://doi.org/10.1093/llc/fqab091
Mastropierro, L. (2018). Corpus stylistics in Heart of Darkness and its Italian
translations. Bloomsbury.
O’Brien, S. (2022). How to deal with errors in machine translation: Postediting. In
D. Kenny (Ed.), Machine translation for everyone: Empowering users in the age
of artificial intelligence (pp. 105–120). Language Science Press. https://doi.org
/10.1080/0907676X.2021.1905673
Oeser, H.-C. (2020). Duel with DeepL. Literary translator Hans-Christian Oeser
on machine translation and the translator’s voice. Counterpoint 4, 19–123.
https://www.ceatl.eu/wp-content/uploads/2020/12/Counterpoint_2020_04
_article_05.pdf
Rayson, P. (2019). Corpus analysis of key words. In C.A. Chapelle (Ed.), The
concise encyclopedia of applied linguistics (pp. 320–326). John Wiley. https://
eprints . lancs . ac . uk / id / eprint / 140803 / 1 / Rayson _ 2019 _ Cor p usA n aly s iso
fKeyWords_Submitted.pdf
Rybicki, J. (2012). The great mystery of the (almost) invisible translator: Stylometry
in translation. In M. Oakes & M. Ji (Eds.), Quantitative methods in corpus-
based translation studies (pp. 231–248). John Benjamins. https://doi.org/10
.1075/scl.51.09ryb
Saldanha, G. (2011). Translator style: Methodological considerations. The
Translator 17(1), 25–50. https://doi.org/10.1080/13556509.2011.10799478
Saldanha, G. (2014). Style in, and of, translation. In S. Berman & C. Porter (Eds.),
A companion to translation studies (pp. 95–106). John Wiley. https://www
.researchgate.net/publication/301104802_Style_in_and_of_Translation
Mark My Keywords 87
Scott, M. (2022). How key words are calculated. WordSmith tools help. https://
lexically.net/downloads/version_64_8/HTML/keywords_calculate_info.html
Scott, M. & Tribble, C. (2006). Textual patterns: Key words and corpus analysis
in language education. John Benjamins. https://www.researchgate.net/
publication/238737537_Textual_Patterns_Key_Words_and_Corpus_Analysis
_In_Language_Education
Taylor, C. (2018). Similarity. In C. Taylor & A. Marchi (Eds.), Corpus approaches
to discourse: A critical review (pp. 19–37). Routledge. https://doi.org/10.4324
/9781315179346
Toral, A. (2019). Post-editese: An exacerbated translationese. In Proceedings of
Machine Translation Summit XVII: Research Track (pp. 273–281). European
Association for Machine Translation. https://aclanthology.org/W19-6627
Castilho, F. Gaspari & S. Doherty (Eds.), Translation quality assessment: From
.1803.08409
Winters, M. (2007). F. Scott Fitzgerald’s Die Schönen und Verdammten: A corpus-
based study of speech-act report verbs as a feature of translators’ style. Meta
52(3), 412–425. https://doi.org/10.7202/016728ar
Winters, M. (2009). Modal particles explained: How modal particles creep into
translations and reveal translators’ styles. Target 21(1), 74–97. https://benjamins
.com/online/target/articles/target.21.1.04win
Winters, M. (2015). Hans-Christian Oeser: Translation strategies of a literary
translator. In S. Egger (Ed.), Cultural/literary translators: Selected Irish-German
biographies II (pp. 210–224). Wissenschaftlicher Verlag Trier.
Woolls, D. (2021). Tetrapla plus. CFL Software Ltd.
Youdale, R. (2020). Using computers in the translation of literary style. Routledge.
https://doi.org/10.4324/9780429030345
Part 2
Machine Translation
Applications in Literary
Translation

4 MT and CAT
Challenges, Irrelevancies, or Opportunities
for Literary Translation?
James Luke Hadley
4.1 Introduction
Although technology explicitly created for the facilitation of translating
texts has a history that is virtually as old as Translation Studies as an
academic field of study, both having developed in the wake of the Second
World War (Tymoczko, 2018, p. 156), the question of whether and to what
extent such technology holds any benefit for the translation of specifically
literary texts has been largely moot for much of that time. Until recently,
Machine Translation (MT) technology had not reached the point at which
literary texts could be considered seriously as points of analysis, and
Computer-Assisted Translation (CAT) tools were similarly rudimentary
from the perspective of operations key to literary translation. Meanwhile,
literary translation has come to be questionable in multiple contexts in
terms of its commercial sustainability, particularly from the perspective of
paying its translators fairly (Taivalkoski-Shilov, 2018, p. 695).
Here, ‘literary translation’ refers to the translation of texts that are
engaged with primarily for aesthetic, rather than functional, reasons (see
also Section I.1, Introduction, this volume). Functional texts, such as
instruction manuals, legal statutes, and recipes describe phenomena and
facilitate actions in the physical world. Literary texts, on the other hand,
frequently evoke or depict the physical world, but the reactions they elicit
in readers are primarily emotional rather than physical. Naturally, literary
texts may have perlocutionary, educational, or informative effects in the
same way that sculptures, paintings, pieces of music or any other artistic
artefact may have. However, these effects are generally secondary to, or
caused by a reaction to, the aesthetic qualities of the texts. Thus, while
advertisements have the underlying perlocutionary aim of encouraging
observers to purchase the products or services being advertised, the trigger
encouraging them to do so is the aesthetic, and thereby emotional, reaction
to the text itself rather than a functional message such as ‘Step 1: select
product. Step 2: pay for product’.
DOI: 10.4324/9781003357391-7
92 James Luke Hadley
The term used here for the translation of functional texts is ‘technical
translation’ in order to avoid confusion with the functionalist school
of translation, which is not exclusive to functional texts (Snell-Hornby,
2006, p. 162). Technical translators have been able to increase their
efficiency dramatically with the use of CAT tools, producing many times
more translated words than they could have without them, and can now
make use of MT functionality to assist in the translation process itself,
making technical translation not only a viable commercial activity but
a lucrative one for the highly skilled. Literary translators, on the other
hand, have certainly benefitted from many aspects of technology, including
easily searchable online dictionaries and corpora, digitised archives, and
word processing technology. However, generally speaking, they have
not historically benefitted from technological tools that are specifically
designed to assist them in the translation of literature (Ruffo, 2022, pp.
26–29). The economics of literary translation are also very different from
those of technical translation. In many contexts, national bodies such as
cultural outreach centres subsidise the translation of national literature
into the languages of the world because the amounts of money consumers
are accustomed to paying for literary works generally do not differ
significantly between translated and non-translated works. Therefore, the
additional work associated with translating a text into another language
requires a subsidy. Alternatively, translators can be paid the equivalent
of a tiny amount per hour for completing the work, effectively turning
literary translation into a hobby, and at the same time preventing it from
being a viable career choice per se for many.
Before CAT tools, including translation memories (TMs), electronic
dictionaries, and project-management software, translation in general was
a labour-intensive, expensive activity. Logistics effectively limited how
much material could be translated each year. As the world has gradually
become ever more interconnected since the end of the Second World War
though, translation has played an ever more important role in facilitating
communication. Without the assistance of CAT tools, it would be unfea-
sible for the same amount of material to be translated as we see today.
Possibly because necessity is the mother of invention, the technology, the
translators, and the technical-translation industry have continually shifted
to keep pace with demand. Many times more translation work can now be
expected and delivered by a single translator than was previously imagi-
nable. Thus, translators without access to such tools can quickly be priced
out of the market. In the third decade of the twenty-first century, it is not
unreasonable to assert that many professional technical translators mak-
ing no use of CAT tools would struggle to remain competitive in terms of
output and may be severely limited in terms of the projects they are able
to take on since so many require the use of specific TMs or termbases.
MT and CAT 93
In these cases, the use of company-specific TMs and termbases enables

consistency to be maintained across translators and ensures that technical
terminology is accurately rendered in translations.
However, while technical texts, especially of the most formulaic types,
written in global languages that lend themselves particularly well to
CAT tools and MT are, perhaps, at the forefront of this shift towards
mechanisation, literary translation sits at the opposite end of the spectrum,
its workflow having developed very little in technological terms since the
1980s. While CAT tools can be used with some forms of literary translation,
particularly works of narrative fiction, there is resistance on the part of
some translators to doing so for a variety of reasons (Ruffo, 2021, p. 126).
In terms of development, too, progress in creating literary MT systems has,
to date, been hampered by the highly flexible and sometimes idiosyncratic
nature of literary texts (Taivalkoski-Shilov, 2018, p. 695–696).
Nonetheless, literary translators may be concerned that their role
will soon be supplanted by machines. This apparent divide between MT
technology and literary translators has historically made it appear as if
one is simply irrelevant to the other (see also Way, Rothwell & Youdale
(forthcoming)). However, recently a new generation of researchers has
begun to emerge, both in Translation Studies and in Computer Science,
who are asking new questions about this apparent divide and challenging
it. Thus, it may be that the divide is shrinking and will soon vanish. The
aim of this chapter is to assess CAT tools and MT systems from the
perspective of literary translation, asking whether they are best considered
irrelevancies, challenges, or opportunities.
4.2 Irrelevancies
For many years, from the perspective of MT and CAT, literature was
simply irrelevant. Similarly, from the perspective of literature, MT and
CAT tools were irrelevant. MT had originally surfaced as a development
of cryptography during the early years of the Cold War when the USA and
its allies had a particular interest in understanding the contents of Russian-
language communications (see Harper, 1955, p. 41; Lennon, 2014, p. 137).
These machines belonged to the same tradition as the early computers that
had been put to work by codebreakers during the Second World War to
decode enemy transmissions (Gambier, 2018, pp. 132–133). The principle
underpinning them was that a mechanical approach to the deciphering of
cyphers, or indeed whole languages, could yield quicker, more reliable,
and less easily problematised results than the comparatively messy, highly
time- and resource-intensive manual approaches that made use of the
human brain. This principle correlates with Saussurean linguistics, under
which paradigm arbitrary lexical signs are used as signifiers and can be
exchanged for others without a loss of underlying meaning (De Saussure,

1916, p. 113).
Thus, when the proof of concept that a machine could indeed be
programmed to translate a set of Russian sentences into English was
made in the famous Georgetown-IBM experiment of 1954, the door was
enthusiastically flung open for the technology to be developed. For those
invested in the experiment, the problem of translation, if mechanised,
would be solved in the foreseeable future:
Linguists will be able to study a language in the way that a physicist

studies material in physics, with very few human prejudices and pre-
conceptions… The technical literature of Germany, Russia, France,
and the English-speaking countries will be made available to scientists
of other countries as it emerges from the presses
(Macdonald, 1954, p. 8)
Despite the mention of ‘literature’ in this quotation, it is important to

remember the overwhelming focus on technical texts in this context. The
military minds with an interest in deciphering the communications of the
Soviet forces were not primarily concerned with poetry or fiction, but with
intelligence that could influence the outcome of the Cold War.
In 1966, when the ALPAC commission observed that the meteoric
improvement in translation technology that had been anticipated was still
not forthcoming, it was reasoned that the funding and research associated
with MT in the USA would be better spent on other translation-related
activities that were more likely to yield practical results in the short term
(ALPAC, 1966, p. 16), though MT research continued in other countries.
The commission’s report called for endeavours including finding ‘means
for speeding up the human translation process’, the ‘adaptation of existing
mechanised editing and production processes in translation’, and the
‘production of adequate reference works for the translator, including the
adaptation of glossaries that now exist primarily for automatic dictionary
look-up in Machine Translation’ (ALPAC, 1966, p. 34). In essence, the
report described rudimentary CAT tools. And it was to the development
of these that research funding was ultimately redirected over the following
years. However, here too, the top priority in terms of text types was not
literature, but the kinds of technical texts that could offer tangible benefits
to the translating culture broadly speaking, especially in the fields of
science and technology that could give the translating culture a strategic
edge over the opposition.
Thus, CAT tool systems and MT systems alike were built overwhelmingly
with technical texts in mind. The underlying aim of the systems was to
convey observable meaning in the sense of the practical applications of
MT and CAT 95
the texts’ contents, as accurately as possible, rather than to manifest style,

perlocution, or emotional reaction, as might be the case if the focus had
been texts which exist for primarily aesthetic reasons, such as literature.
In the context of technical texts, such as legal statutes, contracts,
instruction manuals, or guidebooks, value is tied primarily to texts’
functional relationships with the tangible world. If an instruction manual
only provides helpful instructions, it is valuable. If it only provides
unhelpful instructions, it is not. The same is not true of novels, poems,
and other forms of primarily aesthetic literature. While one may derive
life lessons, inspiration, or even practical ideas from a novel, these could
be seen as incidental to the act of reading the novel. Put another way,
there are most likely more efficient, and direct ways to acquire practical
knowledge than by reading a literary account.
Therefore, in the case of technical texts, it is reasonable to prioritise
the value-adding aspects of the text in translation, and hence, to make
the translation as efficient as possible in conveying the text’s functional
contents. In other words, the priority is creating translations that allow
people speaking the target language to do the same things that the people
speaking the source language can do. Thus, it is reasonable to divide
texts into segments, the translations of which can be reused if they occur
multiple times. It is arguably also reasonable to calculate translators’ fees
based, not on the texts as a whole, but on the segments, and specifically
on those segments for which the translator was required to translate the
segment afresh. Furthermore, it is reasonable to maintain consistency as
much as possible by reusing the same terminology throughout and across
texts.
However, when it comes to literary translation, it could be argued
that segmenting texts in such a way is not only unlikely to embody an
improvement in terms of the translator’s efficiency, but also that it may
actually reduce efficiency in cases where similar or identical segments need
to be translated differently, where the segments in the source language do
not apply meaningfully in the target language, or where textual factors
like paragraph ends are made less obvious by sentence-level segmentation.
Similarly, the underlying principle guiding the translation of literature is
not the tangible-world applications of the knowledge contained within the
texts. And in literature, style certainly cannot be thought of as occupying a
lower priority than this knowledge.
Thus, the workflows on the two sides of the divide were not historically
compatible. The tools being developed were efficient at a set of tasks. But
this set of tasks was specific to technical texts and only marginally applicable
to the tasks actually undertaken by literary translators. Meanwhile, many
of the tasks that are important to literary translators, such as identifying
puns, rhyme schemes, and metaphors (see Chapters 8 and 9, this volume
and Kolb and Miller, 2022), are not included in the functionality of such
tools or may be impeded by the assumptions underpinning the tools.
Therefore, one can be seen as something approaching an irrelevance from
the perspective of the other.
4.3 Challenges
In more recent years, as the quality of MT outputs has improved and CAT
tools have become further integrated into the technical translator’s work-
flow such that clients could insist on the use of specific termbases or TMs,
the attitude that MT systems and CAT tools in general are simply irrel-
evant to the production of literary translations has begun to give way to a
sense that MT systems and CAT tools might pose an existential threat to
the human translator’s livelihood.
Partly because it is a labour-intensive, poorly paid endeavour, trans-
lators working in literary translation often do not do so exclusively
(Wechsler, 1998, p. 22). Many also work as technical translators and have
seen the effects of technology’s arrival on the technical-translation industry
first-hand. They have seen that the advent of CAT technology has tended
to increase the pace and urgency of the industry as a whole, has shifted the
focus of translation away from whole texts towards individual segments,
has required translators to invest in the correct software, and in some
cases has introduced a greater proportion of editing and post-editing to
the workflow of translators than had previously been the case. At the same
time, literary translators have observed the gradual creep of MT systems
into fields of technical translation, such as legal and medical areas, which
have historically been lucrative because of the specialist knowledge they
require, and observed how MT systems have been able to cope relatively
well with the formulaic text types and terminology that are often found in
these fields (see, for example, Wiesmann, 2019).
The concern among literary translators who observe these developments
may be about what the literary translation industry might look like if it
were to come under the same or similar pressures after the introduction
of technology. They may imagine a world in which novels and poems are
similarly translated by the segment, where some segments are pre-filled
based on other works of literature that have already been translated, or
where their role as translators shifts to quality control for the outputs of
MT systems. They may imagine the already meagre fees associated with
literary translation being squeezed even more by unscrupulous editors
counting pre-translated segments with an eye on the bottom line.
Thus, in every respect, if the only translation technology currently in
existence which is made for the translation of technical texts is anything to
go on, the introduction of technology to the literary-translation industry
could be perceived as a threat to the workflow, livelihood, and job
MT and CAT 97
satisfaction of professional literary translators. The financial remuneration

associated with literary translation already makes it barely or not even
viable as a standalone career path in many contexts around the world
for all but the most well-established translators (Wechsler, 1998, p. 22).
Therefore, this remuneration is likely not the primary reason for many
translators to undertake the work. Instead, the primary motivation for
many may be associated with the love of the process and the love of the texts
in question. If the process were undermined, such that the translator’s role
was reduced to translating disembodied segments or checking the outputs
of a machine, much of this motivation may be lost, leaving translators with
a similar choice to those working in technical fields who did not or could
not keep up with the advance of CAT tools: either work in the way that
you like as a hobby, accepting you cannot be commercially viable, or find
something else to do.
However, many if not all these extrapolations are built on factors that
are true in some, and not necessarily all fields of technical translation,
but may or may not hold true in the context of literary translation. In
both cases, the driving factor from the perspective of the industry is the
product, since whatever enjoyment the translator may derive from the
process is incidental to the reasons a party might pay for rendering a
text in translation. In the case of technical translations, if translations are
processed on the segment level rather than the text level, it is also logical
that the remuneration would also proceed on the same level.
However, it is reasonable to ask whether the same holds true in the
case of literature, whether basing the translation process on the sentence-
level segment is meaningful in the context of literature, and whether it is
fair to assume that identical segments in different texts can be translated
identically. Some literary translators may argue that none of these
assumptions holds for them, and therefore, if the assumptions were to be
imposed by inflexible technology or clients, literary translation work may
be problematised by adding unnecessary steps and ultimately embodying a
challenge to literary translators’ way of life.
Ultimately, if MT systems were to become able to translate literature
in a manner indistinguishable from a human translator, it would come to
embody an existential challenge to the livelihoods of literary translators in
general precisely because the margins are already so small in the literary
translation industry.
4.4 Opportunities
It is easy in an already stretched industry to assume that any change will
necessarily constitute a challenge. Thus, what risks being missed in discussions
related to the introduction of technology to a sector which has traditionally
tended towards manual approaches is the potential opportunities associated
with augmenting human workflows with tools that have not previously
existed. While it is not true to say that literary translation during the third
decade of the twenty-first century has the same relationship with technology
as it had twenty years before, it is true to say that the scale and nature of
technology’s impact on the workflow of the translators involved is very
different to those working in many forms of technical translation.
In many contexts, and in the case of much technical translation, it has
become difficult or even impossible for a professional translator to survive
without the use of CAT tools of one form or another. Particularly in the
case of multinational companies and those for whom business pivots on
translation, TMs, and the ability of a translator to make use of them have
become prerequisites to maintain consistency (Taivalkoski-Shilov, 2018,
p. 691). Translators in such situations are routinely sent TMs or termbases
which must be used (ibid). These tools substantially increase productiv-
ity on the part of translators (ibid.). They systematise and speed up the
process of translating by segmenting texts and taking formatting out of
the equation. They prefill segments for which a similar segment exists in
the memory and suggest translations for terminology already listed in the
termbase (Brkić et al., 2009, pp. 354–355). Viewed from the companies’
perspective, the technology also increases efficiency by reducing or obviat-
ing the reproduction of work in the form of retranslating already translated
segments (Taivalkoski-Shilov, 2018, p. 691). Ultimately, this functionality
also creates the opportunity for the companies to separate the segments
which have been translated by the human from those prefilled by the CAT
tool and adjust the fee paid to the translator accordingly. Thus, the trans-
lator’s fee is no longer tied to the length of the document itself, but to the
fraction of that work for which their human input has been indispensable
(Gouadec, 2007, p. 18).
As already noted, this model tends to work on the assumption that
the translation provided for any given segment in one document could be
translated in a similar or identical way if it appeared in another document
or in a different part of the same document. In many contexts, particularly
in the context of technical translation, this assumption may be relatively
safe. However, even in the context of technical documents, it does not
always hold true and has led to the so-called ‘peep-hole’ phenomenon,
whereby translators’ only access to the text they are involved in translating
is through individual or disembodied segments (Melby et al., 2014, p. 671).
In the case of literature especially, working this way may lead to additional
work. Sentences that are structurally similar in the source text may have
substantially different perlocutionary functions from one another because
of the different contexts underpinning different texts. Thus, translators
may be forced to rework inappropriately reproduced segments rather than
translate them afresh.
MT and CAT 99
Nonetheless, recent years have seen literary translators working with

CAT tools, though the efficiencies created by the tools may only overlap to
some extent with those observable for technical translators (Taivalkoski-
Shilov, 2018, p. 691). In the translation of dramatic scripts, for example,
where character names are repeated very frequently in the text and dia-
logue tends to keep segments relatively contained, the use of CAT tools
can speed the process substantially as well as reduce the risk of human
error. The same is also true in the case of literary prose, particularly in the
case of novels with substantial character lists, and especially if the names
of the characters are meaningful, complex, or are themselves translated.
Translators may choose to add these character names to termbases, in
order to simplify the process of reproducing them, to remove the opportu-
nity for human error, and to speed the process of typing them, particularly
in the event that they are long or complex. The same is true in the case
of highly elaborate fantasy (see Chapter 7, this volume) or science-fiction
series, containing extensive lists of technical vocabulary that must be kept
consistent over multiple volumes, potentially translated by different indi-
viduals. In these cases, termbases can prove useful.
However, these uses rely on applying technology that was never
intended for literature to literature as a workaround. They do not address
a majority of the key features which challenge translators working with
literary texts. Features which may weigh heavily on a literary translator’s
mind when working with literary prose may include sentence length,
punctuation, style, irony, puns, intertextuality, direct and reported
speech, the explicitation of culture-specific terms, dialect, and register.
When working with poetry, literary translators may also need to devote
substantial thought to juggling concerns such as rhythm and metre, rhyme,
alliteration, simile and metaphor, assonance, lineation, diction, and
allusion. At the time of writing, the first literature-specific CAT tools are
beginning to appear. PunCAT1 is a good example of such software which
assists translators in identifying and translating the puns in the literary
texts they are working with (see Kolb and Miller, 2022).
Naturally enough, not all literary forms are common to all languages
across the globe, and norms within literary forms are not necessarily
common across languages. Moreover, it is not the case that literary
translations are only valid if they attempt to reproduce the same stylistic
effects as their source texts. It is the case that translations of literary texts
may seek to explain the source text, rather than replace it for the target
culture, as is the case for many Classical works (see e.g., Loeb classical
library).2 It may even be that a literary device used heavily in the source
1 www.ofai.at/software/puncat.
2 www.hup.harvard.edu/collection.php?cpk=1031.
language does not exist in the target language, and therefore, that the
translation will necessarily need to do something else.
This latter case can be illustrated by the idea of translating Shakespeare’s
sonnets into Japanese. Shakespeare’s sonnets belong to the English son-
net tradition which, like those of other European languages, hinges on its
strict use of metre and end rhyme. It makes use of iambic pentameter, a
term indicating five unstressed syllables interspersed by five stressed syl-
lables on each line and a pattern of three alternately rhymed quatrains
terminating in a final couplet, meaning that the ends of each of the lines
rhyme in the pattern ABAB CDCD, EFEF, GG (Walters et al., 2016, p. 5).
Unlike English and ancient Greek, Japanese does not function on the basis
of iambs, or patterns of stressed and unstressed syllables. Neither does
Japanese function on the same basis of syllable count as languages such
as French and Spanish. Rather, the basic unit of the Japanese language is
the mora, which is somewhat comparable to a syllable. It includes either a
consonant and a vowel, a vowel only, or a terminal ‘n’ only (Poser, 1990,
pp. 78–79). Although the pitch and occasionally tone of each mora shifts
according to the word’s pronunciation and position in the sentence, the
stress is constant (ibid).
It would be an overstatement to claim that the translation of rhyme
is impossible in Japanese. However, the structure of the language, where
verb endings are highly regular and fall by default at the end of sentences
has hampered the development of a tradition of end rhyme in poetry
(Chamberlain, 2013, pp. 29).
Thus, what are arguably the two defining features of the English sonnet
as it exists in Shakespeare's source texts would not generally be present in
Japanese translations. Does this mean that the texts cannot be translated at
all? Certainly not. But it does mean that a translator’s judgement to focus
on other elements of the text than these key features is both fully justifiable
and indeed expected.
Over the coming years, we will likely see the development of a suite of
individual tools that aim to facilitate the work of translators of literary
texts. It would be relatively simple for such tools to automatically identify
features such as rhyme patterns (see Chapter 8, this volume), syllable
counts, and sentence lengths in both source texts and draft translations as
a means of deciphering the formal features of the source text in question
in the first place and then as a means of assisting the translator in checking
the translation they have produced. CAT tools already routinely check for
formatting tags, such as hyperlinks or quotation marks present in source
texts but not in translations so as to flag these as possible errors. CAT tools
focused on literary translation could apply a similar approach to other
features, including, for example, rhymes. However, in addition to being
maximally useful to literary translators, such tools would need to include
MT and CAT 101
the added flexibility of not only attempting to enforce what exists in the
source text or what is theoretically possible in a target text, but also what
the translator’s chosen strategy is with regards to the work of literature.
In a similar way that CAT tools in the domain of technical translation
have developed over time, not only in terms of quality but also in terms
of functionality, it may be that translation technology that is particular
to the translation of literature will develop in an iterative manner. Since
observing, identifying, categorising, highlighting, and matching such
formal features in texts is not a radical departure from functionality already
relatively common in text-based applications, most notably, perhaps, in
existing CAT applications, it could be that these will embody some of the
first steps in creating translation tools specifically for literary translation.
A subsequent step may be to integrate some MT functionality. This could
give gloss translations of individual terms found in source texts. Next,
such functionality may be extended to give a range of rough translations
for whole segments, with the aim of prompting the human translator’s
creativity. An early version of this functionality already exists in some
CAT systems, such as Phrase TMS (formerly Memsource Cloud), though
it is not intended specifically for literary translation.3 Later, the systems
used for identifying and categorising formal features could be modified to
inform the outputs of the MT system, such that lists of suggestions could
be given to the human translator of solutions that reproduce any specified
textual pattern such as rhyme. Part of this stage would probably include
functionality that would allow the translator to rank the list of identified
formal features in terms of priority since translation often involves
balancing many competing interests.
A later stage would be to shift the relationship between the technology
and the human slightly, such that instead of the machine making suggestions
to the human for solutions to isolated translation problems, the human
may begin setting parameters for the MT output which they could then
postedit. It is unlikely that TMs of the kinds seen in technical translation
would prove useful in the context of literary works since style can be so
idiosyncratic on the one hand, and such a large part of the appeal of the
work on the other. None of the increments would remove agency from
the literary translator altogether, since the human would be required to
select the parameters underpinning the translation throughout and would
ultimately be needed to postedit the output.
An evolution of this kind would bring literary translation to a similar
stage of technological development that some areas of technical translation
find themselves in at the beginning of the third decade of the twenty-first
3 https://phrase.com/.
century. It is clear that the impact on the industry as a whole would likely
be as profound as the impact of the gradually advancing CAT tools has
been for technical translation. Each increment would hold the potential
to increase a translator’s efficiency in terms of the number of words they
would be able to translate in one day. This increased efficiency could mean
that a vastly greater amount of translated literature could find its way
into circulation since the translation of literature could be made financially
viable, where today in many contexts, the amount of work required to
produce a single translated novel can outweigh the revenue that can be
produced by selling it. What is less easy to see at this stage is whether making
literary translators substantially more efficient would have the effect of
saturating the market with translated literature. If it did, this could either
lead to subsidies being increased to match the market-based competition,
or cut altogether, since the current need to incentivise literary-translation
practice would no longer hold. Either way, it seems likely that if literary
translation were made substantially more efficient than it currently is, the
subsidies would be less necessary as the industry as a whole would become
more efficient. Viewed from this hypothetical perspective, the introduction
of technology can be seen as an opportunity.
4.5 Discussion
This chapter set out to ask whether technology can be seen most reasonably
as an irrelevance, a challenge, or an opportunity for literary translation. It
almost goes without saying that all three and many more perspectives are
simultaneously valid. However, the perspectives that tend to characterise
literary translation’s relationship with technology at any given point in
history may correspond to those which are most closely influenced by what
is and what is not technically possible at the time.
In the early years of translation technology, when the overwhelming
focus was on technical texts, and the use of technology was more of a
luxury than a prerequisite, it was, from the perspective of literature, largely
irrelevant. MT systems were not even at the stage of being able to process
technical texts reliably, and while CAT tools were used in some contexts,
the idea of investing in expensive technology to translate literature would
likely have been a relatively extravagant one. Therefore, under this
paradigm, where the technology was manifestly unable to work with most
forms of literature, it is difficult to see it as anything other than irrelevant
from a literary translation’s point of view.
But, as MT systems passed from statistical to neural models in the
second decade of the twenty-first century, the quality of the outputs
increased substantially, such that post-editing came to be conceivable as
part of the workflow of professional translation for more than the most
formulaic texts (Matusov, 2019, p. 11). At the same time, CAT tools were
MT and CAT 103
becoming smarter, allowing for different forms of segmentation for project

management and for integration with MT systems. The marketplace of
CAT tools was diversifying, bringing down prices, and ultimately mak-
ing their use necessary in many contexts to keep up with demand. At this
point, literary translators aware of the relatively swift change in work
practices among technical translators may have begun making inferences
to do with what literary translation might look like if it experienced simi-
lar changes. They would have been aware of the ever-shrinking bottom
line in technical translation, and the ruthless competition that can exist
between translators, particularly in common language pairs. Being also
aware that literary translation was never a lucrative pursuit, they may have
imagined what would be left for them if corporations were able to control
them through technology as they apparently do their technical-translator
counterparts. Under this set of conditions, the easiest characterisation of
technology from the literary translator’s point of view may well be that of
a threat.
However, as research has progressed in very recent years on the aspects
that are particular to literary translation as far as technology is concerned,
it has become increasingly clear that it is vastly overly simplistic to imagine
that one could roll out a slightly modified version of existing MT technology,
CAT tools, or indeed the technical-translation workflow for literature and
have it work in any commercially viable sense. The end results of any of
them would likely be some text. But whether this was a text that anyone
would be interested in paying for is a question worth asking. Whether the
technology would actually increase efficiency is another.
What is beginning to become clear is that technology used in the realm of
literary translation would not merely be a slightly tweaked version of what
exists in the technical realm, because the needs and practices of the two
forms of translation differ based on the texts themselves, the reasons why
those texts exist, the reasons people engage with them, and the techniques
used to translate them. Therefore, to be useful to literary translators,
technology needs to facilitate the work that they do rather than warp what
they do. With this in mind, we are now starting to see the emergence of
new software packages that individually target specific aspects of literary
texts precisely in order to facilitate the work of translators. More efficiency
means that translation projects can be produced more quickly, meaning
that the money literary translators are paid for their work may come to
be more representative of the money they need to survive. Under such a
paradigm one may be more inclined to see literary translation technology
as an opportunity to be embraced by human literary translators.
Attitudes shift gradually and are not only influenced by what is actually
possible but what is believed or feared to be just around the corner. It may
be that there will be a time when human literary translators are a rarity.
However, it may also be that resisting the use of technology per se could
hasten that time rather than hinder it.
References
ALPAC. (1966). Language and machines: Computers in translation and linguistics
(1416). National Academy of Sciences. https://nap.nationalacademies.org/
resource/alpac_lm/ARC000005.pdf
Brkić, M., Seljan, S., & Bašić Mikulić, B. (2009). Using translation memory to
speed up translation process. In Proceedings of the 2nd International Conference
‘The Future of Information Sciences: INFuture2009—Digital Resources and
Knowledge Sharing’, Zagreb (pp. 353–363). Department of Information
Sciences, Faculty of Humanities and Social Sciences, University of Zagreb.
http://infoz.ffzg.hr/INFuture/2009/papers/INFuture2009.pdf
Chamberlain, B. H. (2013). The classical poetry of the Japanese. Routledge. https://
doi.org/10.4324/9781315012384
De Saussure, F. (1916). Cours de linguistique générale. Payot.
Gambier, Y. (2018). Institutionalization of translation studies. In L. d’Hulst & Y.
Gambier (Eds.), A history of modern translation knowledge: Sources, concepts,
effects (pp. 131–142). John Benjamins. https://doi.org/10.1075/btl.142.24gam
Gouadec, D. (2007). Translation as a profession. John Benjamins. https://doi.org
/10.1075/btl.73
Harper, K. E. (1955). Translating Russian by machine. Journal of Communication,
5(2), 41–46. https://doi.org/10.1111/j.1460-2466.1955.tb01099.x
Kolb, W., & Miller, T. (2022). Human–computer interaction in pun translation. In
J. L. Hadley, K. Taivalkoski-Shilov, C. S. C. Teixeira, & A. Toral (Eds.), Using
technologies for creative-text translation (pp. 66–87). Routledge. https://doi.org
/10.4324/9781003094159
Lennon, B. (2014). Machine translation: A tale of two cultures. In S. Bermann &
C. Porter (Eds.), A companion to translation studies (pp. 133–146). John Wiley
and Sons. https://doi.org/10.1002/9781118613504.ch10
Macdonald, N. (1954). Language translation by machine—A report of the first
successful trial. Computers and Automation, 3(2), 2–10.
literature. In Proceedings of Machine Translation Summit XVII: The Qualities of
Literary Machine Translation, Dublin, Ireland (pp. 10–19). https://aclanthology
.org/W19-7302
Melby, A. K., & Wright, S.-E. (2014). Translation memory. In S.-W. Chan (Ed.),
Routledge encyclopedia of translation technology (pp. 662–677). https://www
.routledgehandbooks.com/doi/10.4324/9781315749129.ch41
Poser, W. J. (1990). Evidence for foot structure in Japanese. Language, 66(1), 78–
105. https://muse.jhu.edu/article/452744
Ruffo, P. (2021). In-between role and technology: Literary translators on navigating
the new socio-technological paradigm [Unpublished PhD dissertation]. Heriot-
Watt University. https://www.academia.edu/45534446
Ruffo, P. (2022). Collecting literary translators' narratives: Towards a new
paradigm for technological innovation in literary translation. In J. L. Hadley,
MT and CAT 105
K. Taivalkoski-Shilov, C. S. C. Teixeira, & A. Toral (Eds.), Using technologies

/9781003094159.
Snell-Hornby, M. (2006). The turns of translation studies: New paradigms or
shifting viewpoints? John Benjamins. https://doi.org/10.1075/btl.66
Taivalkoski-Shilov, K. (2018). Ethical issues regarding machine(-assisted)
translation of literary texts. Perspectives, 27(5), 689–703. https://doi.org/10
.1080/0907676X.2018.1520907
Tymoczko, M. (2018). The history of internationalization in translation studies
and its impact on translation theory. In L. D’hulst and Y. Gambier (eds.) A
History of Modern Translation Knowledge: Sources, concepts, effects (pp.153–
169). John Benjamins. https://benjamins.com/catalog/btl.142
Walters, D. (2016). The wit and wisdom of Shakespeare: 32 sonnets made
thoroughly accessible. Rowman & Littlefield.
embrace translation technology. Tradumàtica. (In press).
Wechsler, R. (1998). Performing without a stage: The art of literary translation.
Catbird Press.
Wiesmann, E. (2019). Machine translation in the field of law: A study of the
translation of Italian legal texts into German. Comparative Legilinguistics, 37,
117–153. https://doi.org/10.14746/cl.2019.37.4
5 Retranslating Proust Using CAT,
MT, and Other Tools
Andrew Rothwell
5.1 Introduction
Until recent years, specialised translation tools were assumed to be
unusable, by definition, by literary translators:
There is a general assumption that literary translation is a unique

and non-repetitive practice and that digital tools designed largely to
improve the productivity of non-literary translators have few applica-
tions in the literary domain.
(Slessor, 2020, p. 239)
Computer-assisted translation (CAT) systems, developed to manage

terminology and exploit (near) repetitions in pragmatic texts, were widely
deemed incompatible with literary writing because it generally lacks
repetition and specialised lexis, depending instead on stylistic variation
within general language, cultural reference, and novel expression, to which
conventional CAT would have little to contribute. Machine translation
(MT) was similarly disqualified by its unreliable, linguistically clumsy
output, again rooted in pragmatic rather than literary language use:
linguistic features such as anaphora, figurative language, complex syntax,
and literary attributes such as style, tone, voice, or aesthetics, were assumed
to be beyond the reach of systems that could not even be relied upon to
translate a simple declarative sentence grammatically. Little wonder,
perhaps, that recent surveys of literary translators reveal a widespread
dislike, but also some fear, of such tools:
Among literary translation specialists, this sentiment has traditionally

been expressed with a certain amount of hubris, where computer-
based systems in general are seen as a threat, but one which is kept at
bay by the nature of the material.
(Hadley et al., 2022, p. 8)
DOI: 10.4324/9781003357391-8
Retranslating Proust 107
While resenting any threatened intrusion of translation technology into

their ‘essentially contemplative profession’ (Ruffo, 2022, p. 31) and ‘craft’
(Daems, 2022, p. 56), many literary translators therefore set themselves
apart from their ‘commercial’ colleagues and dismiss it out of hand as more
trouble than it is worth: how could the computer’s perceived inflexibility,
its ‘one-size-fits-all’ solutions, possibly contribute to the complex, multi-
faceted decision-making that literary translation demands?
However, recent developments suggest that such blanket rejection is
no longer appropriate (if it ever was). The advent, around 2016, of neural
MT (NMT) built on neural networks and machine learning, which in
many language pairs produces much-improved output, has led to renewed
interest in its potential literary uses (see e.g. Toral & Way, 2018, and
the Introduction and Section 2 of this book). Equally, the application of
corpus analysis tools to the literary ST has been shown to enable rich and
otherwise inaccessible insights into its structures and semantic networks,
which can lead to the production of a better-informed translation (see
Youdale, 2020 and Chapter 11 of this book). In the special case of literary
retranslation, the topic of the present chapter, real-time reference to an
existing historical translation aligned with the source text in a translation
memory (TM) has been argued to provide a fruitful ‘stereo’ perspective
on the translation task (Youdale & Rothwell, 2022), in which the self-
censorship of E. A. Vizetelly, the Victorian translator of Émile Zola’s
1884 novel La Joie de vivre, exposed by lacunae in the TM, revealed
the work’s cultural and ideological sensitivities for a contemporary TL
audience (Rothwell, 2021) in a way that can inform later retranslations.
Building on that Zola experiment, the present chapter presents a second
retranslation case study that employs a wider range of technologies to test
the applicability to literary translation of the concept, widely canvased
in commercial translation circles, of the ‘augmented translator’ (Lommel,
2018). It explores the notion of the translation environment as ‘an extended
cognitive system involving a human translator, a TM tool, [and] an MT
engine’ (Balashov, 2020, p. 349, emphasis original). It describes an ongo-
ing project to retranslate into English La Prisonnière (Proust, 1923), the
fifth volume of Marcel Proust’s monumental French novel A la recherche
du temps perdu, using input from MT as well as TM and other online
resources in the editing environment of the Déjà Vu X3 Professional CAT
tool (henceforth, DVX) developed by Atril.1 Principal among these elec-
tronic resources is a segment-aligned TM containing the near-contempo-
rary, public domain English version, The Captive (Proust, 1929) by Proust’s
first translator, Charles Kenneth Scott Moncrieff (1889–1930), which has
1 https://atril.com/product/deja-vu-x3-professional/.
108 Andrew Rothwell
been reprinted multiple times, including twice in modern revised editions.

Alongside this historical TM, systematic use is made of NMT input from
the free version of DeepL,2 as well as various dictionaries and thesauri.
After describing the retranslation brief (Section 5.2) and the project’s tools
and methodology (Section 5.3), the chapter presents a worked example
showing how the tools contribute to the translation of a sample sentence
(Section 5.4). It goes on to discuss the cognitive and creative benefits of
this ‘augmented’ mode of working for a literary retranslator (Section 5.5)
before concluding with some recommendations for future enhancement of
the tools (Section 5.6). Although the case study is of a retranslation pro-
ject, much of the methodology and the conclusions are no less applicable
to first-time literary translation.
5.2 Retranslation Brief
Until the publication of the new translation of the complete In Search
of Lost Time by Penguin Books, including Carol Clark’s acclaimed The
Prisoner (Proust, 2003), Scott Moncrieff’s version was (and for many
readers still is) the voice of Proust (1871–1922) in English. The reservation
most often expressed about his translations is that they changed Proust’s
tone, making him sound precious, over-ornate, and affected: in his essay
‘Proust in English’, for instance, John Sturrock writes of ‘the unhappy way
in which Scott Moncrieff contrived to play down the stringent intelligence
of his author by conveying it in an English prose that is constantly looking
to prettify’ (Sturrock, 1998, p. 115). His English seems to want to live up
to the conventions of Edwardian novel-writing, systematically preferring
‘high-style’ (less common) lexical options (e.g. ‘upon’ for ‘on’, ‘behold’
for ‘see’, ‘impart’ for ‘give’, ‘presently’ for ‘soon’, ‘notwithstanding’ for
‘despite’, ‘inasmuch as’ for ‘since’, ‘falsehood’ for ‘lie’, etc.) and using
occasional archaisms (e.g. ‘he would fain’, and ‘shew’ systematically for
‘show’). Proust’s sentences are often long (the longest in La Prisonnière
being over 400 words, but there are many around a quarter of a page in
length) and always carefully crafted, with intricately balanced and inter-
related clauses which can make it seem as if he is aiming for a logician’s
exposition of a given issue. This is an impression Scott Moncrieff tends to
promote by the explicitation and amplification of connections that Proust
leaves implicit. One aim of the new English cycle is to strip away the
stuffiness of his translation and produce a tighter, lighter, more confidential
and modern version, conveying the first-person narrator’s voice as he muses
ironically, at times amusingly, on his life and times rather as a stream of
2 www.deepl.com/translator.
consciousness (albeit a subtly articulate, controlled consciousness) than as

a conventional exposition.
Since the retranslation brief required such significant divergence from
Scott Moncrieff’s translation ethos and practice, the TM containing his
historical version can only very rarely be used in the conventional CAT
manner, as the source of ready-made translation matches for Proust’s
source sentences. However, seeing each sentence of his canonical translation
displayed automatically on screen by the CAT tool allows the retranslator
to compare it with Proust’s formulation and pay close attention to the
different effects created by both. This ability to evaluate its emphases and
de-emphases, felicities and occasional misreadings, tone of address, and
assumptions about the target audience’s knowledge and values—in short,
the translation strategy of its British upper middle-class author working in
an earlier century, when Proust was an unfamiliar, exotic voice from the
Continent whose radical writing had to be mediated to a select, educated
audience more familiar with the novels of Joseph Conrad and Thomas
Hardy—helps the modern translator to define, by contrast, their own
target audience and approach. The main function of the historical TM is
therefore to act as a point of comparison, making constantly visible the
status quo of Proust in English against which the new translation needs to
define itself.
5.3 Translation Tools and Methodology

The first step in the project was to acquire electronic texts of La Prisonnière
and Scott Moncrieff’s The Captive, both freely downloadable from Project
Gutenberg.3 These were opened in Microsoft Word and, to avoid incorrect
segmentation after the full stop, common abbreviations such as ‘M.’ and
‘Mlle.’ were replaced by the full forms (‘Monsieur’, ‘Mademoiselle’). For
ease of manipulation, both files were then split at corresponding points
in the text into 20 smaller files of around 20 pages each, and each pair
of files was aligned at segment level using DVX’s alignment module to
create reference TMs, numbered 1–20. Before translation could begin,
each of the ST files then had to be reconciled with the text of the reference
Pléiade edition (Proust, 1988). This was necessary because the author
died before he was able to see La Prisonnière through the press, so the
first edition was assembled from typescripts and manuscript fragments
by his younger brother, Dr Robert Proust, and his friend, the man of
letters Jacques Rivière, many of whose editorial decisions were modified
by subsequent scholarship (Proust, 1988, pp. 1628–30). Page numbers
3 The 1923 edition of the ST from www.gutenberg.org/files/60720/60720-0.txt, and the

1929 TT from http://gutenberg.net.au/ebooks03/0300501.txt.
110 Andrew Rothwell
were also inserted between <angled brackets> at this stage to facilitate

electronic searching of the ST, and cross-checking with the text and notes
of the Pléiade volume.
5.3.1 CAT Environment and TM (Déjà Vu X3)
A separate Project was then set up in DVX to translate each of the 20

corrected ST sections containing the corresponding Scott Moncrieff TM
(configured for read-only use), a new, initially empty TM into which the
confirmed translation units (TUs) of the new translation (pairs of aligned
SL and corresponding TL sentences, or segments) would be written, and
a third TM, also initially empty, to contain the whole of the developing
retranslation (this to enable concordance searching in the complete text).
The DVX editing environment displays the ST segments for translation, in
text order, in the left column of the editing grid, with space for the translation
in the column in the middle, and TM proposals in the AutoSearch pane
on the right; a scrollable Preview pane runs beneath SL and TL columns,
showing the translation up to the active segment (highlighted), and the ST
thereafter (see Figure 5.1).
Once each SL segment has been translated and the resultant TU con-
firmed (Ctrl + ↓), the edit point moves down to the following segment
which becomes active, the new SL text string is looked up automatically
in the TM and matches are displayed in the AutoSearch grid (lower right),
with any differences between the new SL string and the one retrieved from
the database marked up using a system of tracked changes, showing where
the TL proposal would need to be adjusted to turn it into an exact match
for the new SL segment (see Figure 5.2).
When considering how to handle the active segment, the translator can
choose whether to accept the TM proposal and then edit it to produce the
new translation, or to ignore the TM proposal and either start typing the
translation or paste in some material from a different source and edit that.
In Figure 5.1, the first two confirmed TUs started out as TM proposals, as
shown by their dark green central bar (indicating an exact, or 100%, TM
match), while the third segment was not based on a TM proposal (light
grey bar). It should be emphasised again that the TM is being used here
in an unconventional way, primarily as a source of TL raw material for
manipulation and modification. By definition, almost every sentence in the
ST has an exact match in the Scott Moncrieff TM (the ones that do not
were either modified or added by the editors in the 1988 Pléiade edition),
and in a commercial environment with a pragmatic ST, the translator could
well be required to accept all such matches unchanged (they may even be
locked to prevent editing); in a literary retranslation, however, that would
of course defeat the whole purpose of producing a new and original TT.
Retranslating Proust
111
Figure 5.1 Déjà Vu X3 editing environment.

112 Andrew Rothwell
Figure 5.2 F
uzzy match of 87% between SL segment and TM segment, with
tracked changes to indicate differences (deletions struck through,
additions underlined).
The creative and cognitive reasons for working with existing TM mate-
rial, and other sources of ‘raw’ text from an NMT engine, dictionaries
etc. rather than starting from scratch will be discussed in Sections 5.4 and
5.5, below. There are also significant practical advantages to working in
a CAT environment such as DVX (other CAT tools offer similar facilities
and advantages) despite its not being designed for this type of translation
(see also Hansen, 2022, p. 51):
• The whole translation task is on the same screen bilingually all the time,
obviating constant shifting of attention between book and computer,
and attendant risk of oversight (e.g. omitting part or all of a sentence).
• ST segments are presented in text order, which helps the translator keep
track of any issues of reference and coherence between segments. DVX’s
real-time scrollable preview pane also allows the work to be read as a
coherent text rather than a sequence of disconnected segments.
• The ST and target text (TT) are locked permanently in alignment,
making it easy to backtrack and check coherence between the new
segment and previously translated material (particularly helpful when it
comes to revising the draft translation, perhaps weeks or months later).
• The TMs can be searched quickly for SL or TL words and phrases

(‘concordance search’), allowing the translator to see how they were
translated previously.
• ST segments can be split or merged, should the translator decide to use
a different sentence structure than the original.
• In-line formats (e.g. italics) can readily be copied across to give the
target segment the same appearance as the source.
• Revision of (any part of) the translation can be carried out incrementally,
at any time, and changes automatically saved to the TM, with both the
ST and original TM proposals visible on screen throughout.
• Data security: since confirmed TUs are automatically stored in the TM
database, in the event of a computer crash the ST can be automatically
retranslated to recover the TT up to that point.
• When the finished TT is exported from the CAT tool, it will automatically
be given the same file format, layout, and appearance as the ST.
These generic CAT benefits combine to make a bilingual editing environment

that is already moderately well adapted to literary retranslation, and indeed
the production of a first translation where there is no prior TM available
despite the fact that many of the more advanced features of such tools are
not needed and they arguably lack others that would make them more
useful for specifically literary purposes (see Section 5.6, below).
5.3.2 Interactive NMT (DeepL, Free Version)
Primarily to assist with translating segments without a useful TM match,

but potentially also to offer an alternative to one or more fuzzy (inexact)
match(es), since the early 2000s CAT tools have offered optional MT
integration via plugin or API key (Zaretskaya et al., 2015, p. 79). In this
conventional hybrid (TM + MT) mode of working, the active segment
is typically sent automatically from the CAT tool to the configured MT
service(s), with the TL results then being brought back into the CAT
environment and displayed alongside any TM matches for possible
selection and post-editing by the translator. It would have been possible to
configure DVX in this way for the purposes of the present project, but that
would have meant missing out on a more recent development in automatic
translation: user interactivity.
A pioneering commercial implementation of interactive MT, the kind
of system hypothesised by Daems (2022, p. 45) to be potentially ‘better
suited to the translation of literary texts than […] regular post-editing
systems’, is found in the Lilt online translation environment.4 Lilt is
4 https://lilt.com/.
114 Andrew Rothwell
designed as an ‘MT-first’ CAT tool in which TM use is optional and every

active segment is submitted to an in-house NMT engine, irrespective of
whether a viable TM proposal is also found. Like other MT systems, Lilt
offers a complete translation of the active segment, but its MT proposals
adjust themselves in the light of user input: if the translator prefers to
start the TL sentence differently from the initial proposal and types an
alternative first word, the rest of the proposal is recalculated on the basis
of that prompt, and so on, word by word, until the translator is happy
with the whole proposal and accepts and confirms it. Lilt’s MT engine is
also adaptive, meaning that it is re-trained on the fly to take account of the
user’s input (‘corrections’), allowing the company to claim that ‘translators
work at superhuman speeds with predictive translation suggestions that
get smarter and more accurate over time’.5 This flexible NMT behaviour
offers a potentially promising approach for literary translators, but when
a small-scale experiment with Lilt was run on a sample of La Prisonnière,
the results were disappointing: perhaps because its training materials were
not appropriate for this task, the baseline engine produced English output
that was lexically and syntactically too flawed to work with effectively.
An alternative source of MT ‘raw material’ was therefore sought.
Among popular free online MT tools, DeepL Translator6 commended itself
because it also offers significant interactivity, although this is lost when it
is connected to a CAT tool in the conventional way via API, with only the
initial static translation proposal being retained. (It should be noted that
in the free version, unlike DeepL Pro, any edits made by the translator are
captured by the software and may subsequently be used as training data
for the MT engine, so it is not suitable for use on copyright, confidential,
or otherwise sensitive material.) DeepL can be used in a web browser,
optionally open on a second screen to avoid constantly having to switch
between windows on a single screen, but it also has a convenient and free
downloadable app (available for both Windows and macOS) which, once
installed, integrates smoothly via keyboard shortcuts with other editing
environments including CAT tools. The active SL segment can be sent
from DVX (or any other CAT tool) to the DeepL app, running by default
in a resizable pop-up window by selecting the text (e.g. by triple-clicking
on it) and typing the (user-configurable) keyboard shortcut CTRL+C+C.
Then, if the translation proposal is found to be useful, it can be pasted
back into the CAT tool (the cursor must first be moved to the correct tar-
get cell in the editing grid) by clicking the app’s ‘Insert to…’ button. The
DeepL pop-up can be positioned adjacent to DVX’s AutoSearch pane (as
in Figure 5.3) to facilitate the comparison of the MT proposal with Scott
5 https://lilt.com/technology/translate.
6 www.deepl.com/en/translator.
Retranslating Proust
115
Figure 5.3 DeepL interactive proposal in a pop-up window alongside a TM proposal in DVX.
116 Andrew Rothwell
Moncrieff’s translation of the same segment from the TM, allowing the
translator to assess the different solutions, decide which one to work with
and paste it in.
Offering a different and arguably even more flexible method of
translator interaction from Lilt, DeepL proposes a complete translation of
the segment but also allows the translator to click on any TL word to open
a drop-down list of alternatives, both lexical and syntactic, which come
from lower-ranked translation proposals generated by the system but
not initially displayed. If one of the alternatives from the list is preferred
and inserted (in the example in Figure 5.3, ‘endearments…’ is about
to be selected in preference to ‘tenderness’, with the option to add that
translation to the user’s Glossary for future re-use), the rest of the proposal
from that point will be recalculated, as in Lilt. Equally, the translator can
intervene at any point in the TL sentence and manually over-type one
or more word(s) with a preferred solution not from DeepL’s list, again
resulting in a recalculation of the rest of the sentence. In combination, these
two ways of interacting with the MT proposal enable a kind of ‘guided
post-editing’ that is quick and easy to implement, so that quite substantial
adjustments can be made before the TL segment is inserted back into the
CAT environment for further polishing.
5.3.3 Dictionaries, Thesauri
The usefulness of DeepL’s app does not end with its interactive MT
proposals. It is also capable of exposing, in the form of a simplified
bilingual dictionary entry, word-by-word equivalence data drawn from
the vast tagged corpora on which the system is trained, and which can also
be accessed via the partner website of the Linguee bilingual concordancer7
(which pre-dates DeepL). In the example shown in Figure 5.3, selecting
the polysemic SL word ‘tendresse’, challenging to the translator because
its psychological extension in French is wider than that of its English
cognate, which also possesses a physical sense that the French word lacks,
generates an item ‘Dictionary entries for “tendresse”’ at the bottom of the
app window that can be expanded to display the data shown in Figure 5.4.
In addition to basic grammatical information, some examples of
usage and links to pronunciation sound files, the list of possible English
translations for ‘tendresse’, with the bottom row classified as ‘less
common’, constitutes a basic thesaurus which can give the translator a
quick overview of some available TL options within the same interface as
the MT proposal.
7 https://www.linguee.com/.
Figure 5.4 DeepL’s SL Dictionary display.
Figure 5.5 DeepL’s TL ‘thesaurus‘ function.
The statistically derived interactive drop-down list generated by clicking

the MT-proposed TL equivalent, ‘tenderness’, can also play the role of a
thesaurus as it contains most of the options in the dictionary entry along
with others less closely related (see Figure 5.5).
Although it should be noted that this list also contains some irrelevant
words (‘tendencies’, ‘tenders’), and even some non-words (‘tenderties’,
‘tenderities’, and lower down, ‘tenderlihoods’), ‘hallucinations’ generated
118 Andrew Rothwell
as unwanted artefacts by the NMT process (Raunak et al., 2021; see also
the Introduction and Chapter 1 of this book), this feature can allow the
translator not just to review near-synonyms, but to select a possible solu-
tion and evaluate how well (or not) it works in the TL draft sentence. This
lexical interactivity makes it easy to experiment with different options and
instantly reverse those that turn out to be unsatisfactory: in this respect,
DeepL’s interactive NMT app can play the role of a sandbox (a meta-
phor borrowed from the field of Computer Science), a safe environment
in which to explore different options and combinations before committing
a preferred version to the CAT editor for further work. It is important
to emphasise that such experimentation is not just a matter of linguistic
juggling: it can be an important part of the cognitive activity of learning
the complexities of the ST by judging the implications, both within and
beyond the active segment, of different options—which in the case of a
stylist such as Proust can be very numerous and complex—and narrowing
them down to a manageable number of avenues for further exploration.
Of course, there are times when the lexical resources of a single tool
such as DeepL are not sufficient for the task at hand, and more complete
or specialised input is needed. For instance, the Thesaurus.com website8
offers a wider range of near-synonyms, often classified by tabs into different
superordinate categories. These are colour-coded from red (most relevant),
then dark or light orange, to yellow, for semantic proximity to the search
term, and clickable to allow exploration down a particular semantic
path, opening a list of the synonyms for one of the words in the original
list. The Trésor de la langue française informatisé online dictionary9
also proved indispensable for digging deeper into the precise meaning of
rare SL words (e.g. the shape and material of the ‘toque’ headgear that
Albertine hankers after (Proust, 1988, p. 572)). It was useful to have these
and other resources permanently open in different browser windows,
despite the attendant screen clutter and the time taken to navigate between
them. Sometimes, too, it was necessary to turn away from the computer
for confirmation of a meaning and consult general and specialised paper
dictionaries. For instance, the revised 1934 edition of Harrap’s Standard
French and English Dictionary (Mansion, 1940) was an essential source of
‘period’ vocabulary relating to technology, fashion, social conventions etc.
(e.g. ‘petit bleu’ (Proust, 1988, p. 838) meaning not, as DeepL accurately,
but in context erroneously suggested, ‘little bruise’, but an urgent message
sent across Paris by pneumatic tube); while the Dictionnaire du français
non conventionnel (Cellard & Rey, 1981) documents the Belle Epoque
homosexual slang associated with the character of the Baron de Charlus,
8 www.thesaurus.com/.
9 http://atilf.atilf.fr/.
and glosses in detail (including an extensive quotation from La Prisonnière)

the obscene expression ‘casser le pot à quelqu’un’ that Albertine is so
embarrassed about almost blurting out, and the narrator takes over three
pages to analyse (Proust, 1988, pp. 840–42).
5.4 A Worked Example

To illustrate the decision-making processes prompted by the ‘extended
cognitive system’ described above, let us now examine in more detail the
segment already shown in Figures 5.1–5.5. It occurs just over a quarter of
the way through the book, in a long passage where the narrator, Marcel,
is obsessively trying to work out whether anything his sleeping girlfriend
Albertine has told him about her activities is actually true. Table 5.1 pre-
sents in parallel text the SL segment (in italics, abbreviated (P)), its transla-
tion by Scott Moncrieff (abbreviated (SM)), and DeepL’s unedited draft
Table 5.1 Translation raw materials for a sample sentence of La Prisonnière.
(P) Proust (1988, p. 621) (SM) Scott Moncrieff (DL) DeepL (unedited)
(1929)
Ses silences n’étaient So her silences were Her silences were therefore
donc que des voiles, merely screens, her only veils, her surface
ses tendresses de surface affection tenderness was only
surface ne faisaient merely kept beneath holding back a thousand
donc que retenir au the surface a thousand memories that would
fond mille souvenirs memories which have torn me apart ―
qui m’eussent déchiré would have rent my her life was therefore
— sa vie était donc heart, her life was full of those facts whose
pleine de ces faits full of those incidents mocking account, the
dont le récit moqueur, the derisive account, laughing chronicle,
la rieuse chronique the comic history of constitutes our daily
constituent nos which form our daily chatter about others,
bavardages quotidiens gossip at the expense about the indifferent,
au sujet des autres, des of other people, but which, as long as a
indifférents, mais qui, people who do not being remains misplaced
tant qu’un être reste matter, but which, in our heart, seem to us
fourvoyé dans notre so long as a person to be such a precious
cœur, nous semblent remains lost in the enlightenment of his or
un éclaircissement dark forest of our her life that, in order
si précieux de sa vie heart, seem to us so to get to know this
que pour connaître precious a revelation underlying world, we
ce monde sous-jacent of her life that, for the would gladly give up
nous donnerions privilege of exploring ours. (88 words)
volontiers la nôtre. (78 that subterranean
words) world, we would
gladly sacrifice our
own. (94 words)
120 Andrew Rothwell
(abbreviated (DL)), to illustrate how comparing the translations with the

original and each other can guide the retranslator’s decisions (the discus-
sion that follows does not assume that the reader knows French).
A full account of the cognitive processes involved is beyond the scope of
this chapter, but some significant differences are immediately visible which
help frame the translator’s choices.
The first is segment length (DVX gives the length of SL and TL segments
in characters, but words would be the more useful unit): SM uses more
than 20% and DL almost 13% more words than P. While translations are
often longer than their originals, not least due to the first three of Antoine
Berman’s ‘twelve deforming tendencies of translation’, i.e. rationalisation,
clarification, and expansion (Berman, 2021 [1985]), this does suggest
some redundancy which may point to one or more underlying translation
issues. SM especially explicates P’s slightly unclear notion of someone
being ‘fourvoyé dans notre cœur’ [lost in our heart] into the unmotivated
full-blown metaphor ‘lost in the dark forest of our heart’ and expands
‘pour connaître’ [to learn about] into the higher-register ‘for the privilege
of exploring’, suggesting in both cases that something more direct might
be achievable. In terms of lexical choice, SM uses ‘rent’, the past-participle
of the rare and high-style verb ‘rend’, rather than DL’s ‘torn’, which may
be preferable in view of the retranslation brief (see Section 5.2, above).
On the other hand, DL’s incomplete syntax in ‘those facts whose mocking
account, the laughing chronicle [of which]’ draws attention to a tricky
syntactic challenge that SM handles successfully, though his lexical choices
(‘account’, ‘history’) are not necessarily preferable to DL’s: these are points
where the retranslator can concentrate on finding a more economical
overall solution.
Another significant issue highlighted by the comparison is the coherence
and cohesion of the surface-depth metaphor (tendresses de surface [surface
tendernesses] and retenir au fond [keep/hold down at/on the bottom]):
by fond, does P mean something beneath the ground like a cave (as in
SM, who opposes ‘surface’, unnecessarily repeated, with ‘subterranean’)
or, as seems more suitable here, the bed of a body of water (a reading
supported by retenir in the sense of [hold down]? Either way, DL’s non-
specific ‘underlying’ is not an effective compromise because it largely
erases a significant metaphor, and it has also selected the wrong sense of
retenir (‘holding back’). Another question that we might not even notice
without the comparison, but which touches on the novel’s homosexuality
theme, is the gender of the missing person (the French possessive sa is
ambiguous): SM erases the ambiguity by opting for ‘her’, taking it as a
reference to Albertine while DL’s ‘his or her’ is more accurate, but from
an inappropriately administrative register. Having been alerted to this
issue, the translator may seek a way of keeping the ambiguity open. These
are just some of the linguistic and textual cruxes highlighted by the kind
of three-way comparison described here, which allows the retranslator
to identify rapidly and work on the high-level challenges posed by the
ST while treating the verbal details of the two solutions as available raw
materials for further exploration (e.g. via a synonym search in DeepL, or a
thesaurus, or a concordance search of the TMs).
In the event, it was judged that DeepL’s version with some interactive
adjustments would require less modification than Scott Moncrieff’s, so
that was the one pasted into DVX and further edited, with the above
considerations in mind to produce the following draft translation:
So her silences were just veils, her superficial endearments merely

kept a thousand memories submerged that would have torn me apart;
her life was full of those incidents, the mocking narrative, the comic
chronicle of which feeds our daily gossip about people who don't mat-
ter to us, but which, when someone has gone missing from our heart,
we feel casts such a precious illumination on their life that to gain
access to its nether world, we'd willingly sacrifice our own. (80 words)
This is not yet a definitive effort; it may well be revised in a second and
final draft, but it does try to offer solutions to the specific problems
identified by comparing P, SM, and DL which are compatible with the new
edition’s objectives of verbal economy, fluency, and relative informality,
emphasising the stream-of-consciousness aspect of Proust’s labyrinthine
meditations rather than their complex logical structure.
5.5 Discussion
In this retranslation case study, the main inputs supplied by current
translation technologies (a historical TM, interactive MT), which often
supplied significantly different material from each other, almost never
offered ready-made translation solutions but were found to save time at
the preliminary comprehension stage to expose through comparison many
of the key translation issues in a given segment, and to provide useful raw
materials for the retranslator’s further exploration notably via a synonyms
search (for confirmation from other literary translators, see Daems, 2022,
p. 53). It is mainly in this sense that the combination of ST, TM, and
MT in a standard CAT tool, in partnership with the retranslator’s own
linguistic and cultural expertise, can be said to constitute an ‘extended
cognitive system’ which, in literary translation perhaps even more than
in more commercial types, both augments the translator’s understanding
of the text’s challenges and expands, in a quickly assimilable manner, the
options available for working on them. A particularly apt metaphor for
the translation environment that results is indeed the sandbox: relieved
122 Andrew Rothwell
of much of the lexical and syntactic spadework normally required

when first approaching a complex sentence unaided, the translator is
free to concentrate on identifying and experimenting with solutions to
higher-level problems such as the coherence of metaphorical structures,
the prosody and rhythm of the sentence, and achieving a consistent
voice by generating different options which can be manipulated until
a workable combination is found, revisable at any time in the light of
future re-reading. At the stylistic level, the tools contribute in this way
to creating a significantly different overall experience for the English-
language reader which may, over time, have an impact on the novel’s
reception.
The demonstrable practical and cognitive benefits that translation
technology can bring counter the claims by some literary translators that
tools would only ‘get in the way’ and hamper their ‘originality’, ‘inspiration’,
or ‘creativity’ (Ruffo, 2022; Daems, 2022, p. 58). It must surely be the case
that, however expert and well-read in source and target languages and
their associated literary cultures, a (re)translator is unlikely to discover by
introspection, even dictionary-assisted, as numerous or as rich possibilities
and connections as can be uncovered by this heuristic, essentially ludic
application of translation tools with appropriate data. Hansen (2022,
p. 52) confirms this expansion of creativity from personal experience:
‘CAT tools helped speed up the translation and research processes, giving
more time to focus on creativity and facilitating the critical study of the
original work’. Kolb and Miller (2022, p. 85) similarly found that their
PunCAT program ‘can effectively support the translation process in terms
of facilitating brainstorming, stimulating creative thinking, providing
inspiration, and broadening the translator’s pool of solution candidates’.
In the present case study, the ability to move easily between and compare
a variety of versions of the same text (ST, historical TT, MT-generated
TT) is casting light on each of them from multiple perspectives (linguistic,
stylistic, cultural, historical, ideological…), providing a rich and informative
context by which the translator’s decisions may be framed. Far from
damaging the literary translator’s originality by reductively directing their
thinking down pre-established paths, as is sometimes alleged (see Ruffo,
2022), current translation tools can liberate and enhance their creativity
by cutting through much of the cognitive tangle associated with making a
first draft, clarifying and ranking the decisions that need to be made, and
allowing concentration on the more stimulating, higher-level challenges.
The draft can then be revised in the same CAT environment, with all the
resources that contributed to the translation of each segment available for
review, before the second draft TT is exported from the system for final
polishing in a standard text editor, in a two-column Word table where it
will be aligned at paragraph level with the ST.
5.6 Conclusion: A Future Literary Translation Environment Tool?

This retranslation project, like the previous one on Zola, makes only lim-
ited use (TM lookup and retrieval, including concordance search) of the
many features offered by a conventional CAT tool optimised for produc-
tivity in non-literary contexts. Such use might even be described as deliber-
ately perverse: ‘computer-assisted literary translation diverges from other
areas, as diversity and choice outweigh precision’ (Hansen, 2022, p. 54).
A literary retranslator is not concerned with pasting in 100% segment
matches from a TM, nor with fuzzy match calculations, pretranslation,
subsegment recall, match repair, or many of the interface features (com-
plex colour coding, WYSIWYG font handling, tag placement routines,
etc.) that CAT tools provide to facilitate the cognitive processing of repeti-
tions in pragmatic texts. Neither has the standard method for integrating
MT into the CAT environment, the presentation of complete automati-
cally translated TL segments in the same list as TM proposals for post-
editing, been found optimally useful. The most productive current resource
for prompting creative solutions, complementing the historical TM, has
turned out to be DeepL’s interactive MT system, along with online diction-
ary and thesaurus websites, all addressed externally to the CAT tool using
a multi-monitor setup with (parts of) relevant hits copied and pasted into
the CAT editing grid.
For the purposes of literary translation, it would seem desirable for
these (and other) external resources to be brought flexibly into the same
editing environment as the TM-searching routines and displayed alongside
proposals from potentially multiple TMs, as internal resources available to
the translator for quick assimilation, selection, and combination, possibly
with common and/or variant textual elements highlighted using a system
of colours. A future computerised literary translation environment might
therefore take the form of a simplified CAT editor with flexible integration
of multiple external resources (MT, corpora of previous translations,
dictionaries, encyclopedias, etc.), which would be displayed in pop-up
windows and remain ‘live’ and interactive, much as the DeepL app
currently does externally. The user would then be able to select all or part
of the active segment, right-click to display a list of configured external
resources, choose one or more of them, send the highlighted string to the
external site(s) and display the returned information in one or more live
pop-up window(s).10 These could be numbered to allow easy insertion of
10 Daems (2022, pp. 58–9) also notes that, although most of her survey respondents ‘did
not want to think about’ how tools might be optimised, several ‘would like to be able to
click a word and get a variety of information from different resources such as (synonym)
dictionaries’.
124 Andrew Rothwell
the information they contain via a quick key combination, in the manner
of TM proposals in a current CAT tool. In this way the translation tool of
the future, rather than adding ever more numerous internal features and
fine-grained routines for extracting information from TMs, would act as
an outward-facing creative portal augmenting the literary translator with
flexible access, within a single interface, to potentially numerous linguistic
and stylometric resources, facilitating the analysis of translation problems
and discovery of creative solutions. In Chapter 14 of this book, Rudan
et al. set out the principles on which an even more ambitious translator-
augmenting CALT tool should be based and present their progress to date
in developing its key components. The opportunity to make an informed
contribution and influence the design of such future tools is one more
reason for literary translators to engage now with the technologies already
available.
Acknowledgements
I am grateful to Atril Solutions and DeepL SE for permission to reproduce
screenshots of their translation tools, Déjà Vu X3 and DeepL Translator
respectively.
References
Balashov, Y. (2020). The translator’s extended mind. Minds and Machines, 30(3),
349–383. https://doi.org/10.1007/s11023-020-09536-5
Berman, A. (2021 [1985]). Translation and the trials of the foreign. In L. Venuti
(Ed. & Trans.), The translation studies reader (4th ed., pp. 247–260). Routledge.
https://doi.org/10.4324/9780429280641
Cellard, J., & Rey, A. (1981). Dictionnaire du français non conventionnel.
Masson.
technology: The role of awareness and attitude. In J. L. Hadley et al. (Eds.),
doi.org/10.4324/9781003094159
Hadley, J. L., Taivalkoski-Shilov, K., Teixeira, C. S. C., & Toral, A. (Eds.). (2022).
.4324/9781003094159
Hansen, D. (2022). The challenges and relevance of computer-assisted literary
translation. In The many faces of translation (pp. 46–62). European Parliament,
Directorate-General for Translation. https://data.europa.eu/doi/10.2861
/653362
Lommel, A. (2018). Augmented translation: A new approach to combining human
and machine capabilities. Proceedings of the 13th Conference of the Association
for Machine Translation in the Americas (Volume 2: User Track) (pp. 5–12).
https://aclanthology.org/W18-1905
Kolb, W., & Miller, T. (2022). Human–computer interaction in pun translation.

In J. L. Hadley et al. (Eds.), Using technologies for creative-text translation (pp.
66–88). Routledge. https://doi.org/10.4324/9781003094159
Mansion, J. E. (Ed.). (1940). Harrap’s standard French and English dictionary. 1
French–English. George G. Harrap & Co.
Proust, M. (1988). La prisonnière. In J.-Y. Tadié, A. Compagnon, & P.-E. Robert
(Eds.), À la recherche du temps perdu (Vol. 3, pp. 517–915). Gallimard.
Proust, M. (2003). The prisoner (C. Prendergast & P. Collier, Eds., C. Clark,
Trans.). Penguin.
Proust, M. (1929). The captive (C. K. Scott Moncrieff, Trans.). A. A. Knopf.
Raunak, V., Menezes, A., & Junczys-Dowmunt, M. (2021). The curious case of
hallucinations in neural machine translation. arXiv. https://arXiv.2104.06683
Rothwell, A. (2021). Uses of alignment: Retranslating Zola with/against Vizetelly.
La Main de Thôt, 9. https://www.researchgate.net/publication/359617463
Rothwell, A. & Youdale, R. (2022). Computer-assisted translation (CAT) tools,
translation memory, and literary translation. In S. Deane-Cox & A. Spiessens
/9781003094159
Slessor, S. (2020). Tenacious technophobes or nascent technophiles? A survey of
the technological practices and needs of literary translators. Perspectives, 28(2),
238–252. https://doi.org/10.1080/0907676X.2019.1645189
Sturrock, J. (1998). Proust in English. In The word from Paris: Essays on modern
French thinkers and writers (pp. 110–118). Verso.
Zaretskaya, A., Corpas Pastor, G., & Seghiri, M. (2015). Integration of machine
translation in CAT tools: State of the art, evaluation and user attitudes. SKASE
Journal of Translation and Interpretation, 8, 76–88. http://www.skase.sk/
Volumes/JTI09/pdf_doc/04.pdf
6 Author-Tailored Neural Machine
Translation Systems for Literary
Works
Antoni Oliver
6.1 Introduction
Literature is one of the areas where machine translation (MT) has not
been used until very recent times. Nevertheless, the impressive increase in
quality obtained with neural machine translation (NMT) systems has led
some authors to start researching the use of MT in literature and other
types of creative texts (see e.g. Toral & Way, 2015a, 2015b, 2018). The
use of source texts and translations of literary texts to create parallel
corpora to train MT systems can lead to copyright issues (see Moorkens
& Lewis, 2019). Many original texts are available in the public domain,
as copyright expires after a given number of years from the death of the
author (between 50 and 100 years, depending on the country’s legislation).1
The same copyright issue applies to translations, making it difficult to find
good translations in the public domain.
In this chapter, a novel method for training NMT systems tailored to
a given author (or a given series of novels, as in our work) is presented.
The method uses the source-language texts of the novels to select parallel
segments from very large parallel corpora. In this way, when training a
tailored system for an author or series in the public domain, no copyrighted
human translations are used.
The goal of the trained MT systems is not the production of translated
versions to be published and read, but the creation of bilingual e-books.
Readers want to read these bilingual e-books in the original language, and
we expect them to access the translated version only to understand difficult
sentences or paragraphs.
1 For a comprehensive list of copyright durations, see https://en.wikipedia.org/wiki/List_of

_countries%27_copyright_lengths.
DOI: 10.4324/9781003357391-9
Author-Tailored Neural Machine Translation 127
6.2 Machine Translation and Literature

The interest in MT for literary texts is quite recent. A good summary of
the previous work in this area can be found in the Introduction of this
volume. In this section, we will focus our attention on previous work on
the training of author-tailored NMT systems (see Chapter 1, this volume),
as well as the use of MT systems for translating literature, with different
goals of producing a translated version of the work to be published, with
or without human post-editing.
The first attempt to train an author-tailored NMT system can be
found, to the best of our knowledge, in Kuzman (2019). In this work,
experiments of translations of novels from English to Slovene using both
Google Translate2 and a tailored NMT system trained with a relatively
small corpus are presented. Although the evaluation results for this
tailored system are worse than those of Google Translate, the study reports
two interesting findings: there are still gains in productivity when using
the tailored system, and the system tailored to a given author obtained
promising results.
One important challenge for MT for literary works is that the writing
conventions differ substantially between authors (see Chapter 3, this
volume), time periods, genres, and forms of literature, and that authorial
styles are not necessarily transferable (Hadley, 2020). This makes it very
difficult to train an author-tailored MT system unless we use the existing
translations of this author’s works. But as Hadley states, what’s the point
of creating an MT system to translate works that have already been
translated? If we did want to do this, assuming that the author might
continue to produce more books that would need translating, the problem
would be that the amount of text needed to train the system is much larger
than the overall production of any author. While building an MT system
in this way may not be profitable, tuning an existing MT system with the
author-specific data might still be.
In most previous work, the goal of MT is to produce high-quality
translations that preserve both the content and the reading experience
(Guerberof & Toral, 2022). But some authors have suggested other
possible uses of MT for literary works. For example, as Matusov (2019)
suggests, MT can be used to translate less popular or new authors, with no
access to the production costs of human translation and regular publishing
processes. This can be an opportunity for some authors to access an
international market, and for readers to get to know new authors and
cultures. If this proves to be popular, some of these works could then be
translated by humans and published in the normal way.
2 https://translate.google.com/.
128 Antoni Oliver
Another use of MT for literary works is the creation of bilingual e-books

as a reading aid for learners of the source language (Oliver et al., 2019).
This use will be further explored in the present chapter.
6.3 The InLéctor Project

InLéctor3 is a collection of freely available bilingual e-books. The goal
of this collection is to promote reading in the original language. In these
e-books, each segment of the original work is linked with the corresponding
segment in the target language. The e-books are published in several
formats (EPUB, mobi, and HTML) and can be read on different devices
(computers, tablets, e-book readers). The InLéctor project publishes only
books in the public domain. There are many interesting books in various
languages in the public domain, but it is very difficult and, in some cases,
even impossible to find a good translation of these works which also
resides in the public domain. For this reason, we are starting to use NMT
systems tailored to literature to produce the translated versions (Oliver
et al., 2022). The user is expected to read most of the book in the original
language and use the machine-translated target-language versions only to
read difficult sentences or paragraphs. In this way, the reading experience
will remain intact, as it comes from the original text.
6.4 Experiments
6.4.1 Source Texts: The Sherlock Holmes Series by Conan Doyle
The work used in this experiment is The Adventures of Sherlock Holmes

by the British writer Arthur Conan Doyle (1859–1930), first published
in 1892. In this work, we find the 12 earliest short stories featuring the
detective Sherlock Holmes, which were published in 12 monthly issues of
the Strand Magazine from July 1891 to June 1892.
There are several other works by Conan Doyle featuring Sherlock
Holmes. The traditional canon of Sherlock Holmes consists of 57 short
stories published in five books (The Adventures of Sherlock Holmes
(1892), The Memoirs of Sherlock Holmes (1894), The Return of Sherlock
Holmes (1905), His Last Bow (1917), and The Case-Book of Sherlock
Holmes (1927)), and four novels (A Study in Scarlet (1887), The Sign of
the Four (1890), The Hound of the Baskervilles (1901–1902) and The
Valley of Fear (1914–1915)).
3 https://inlector.wordpress.com/.
A set of 1,000 segments of one copyright-free Spanish version of The

Adventures of Sherlock Holmes, automatically aligned with the corre-
sponding English segments, was used as an evaluation set.
6.4.2 The CCMatrix Corpus
We need a large, high-quality corpus to select the most useful segment

pairs to train the tailored NMT systems. There are several large parallel
corpora available in the Opus project.4 In Table 6.1 we can observe the
parallel corpora for the English–Spanish pair with more than 5 million
segment pairs, along with the number of segments and tokens for each
language.
For these languages, several of the largest corpora are domain-specific,
as they are created using patent information (EuroPat), United Nations
documentation (UNPC and MultiUN), or European Union documentation
(Eubookshop). We are more interested in general language corpora as it
will be more likely to find similar segments to those from the novels we
want to translate.
Considering all these figures, we decided to use CCMatrix. At this point,
it is worth summarizing the process of creating this corpus performed by
MetaAI (Schwenk et al., 2021). As the authors state, CCMatrix is the larg-
est high-quality web-based corpus for training translation models. It con-
tains more than 4.5 billion parallel sentences in 576 language pairs. This
is, then, a great resource both for high- and low-resource language pairs.
CCMatrix is compiled using several crawls of the Common Crawl public
dataset. Common Crawl5 is a non-profit organization that systematically
Table 6.1 Size of the corpora available at Opus for the English–Spanish language
pair with a size larger than 5M segments.
Corpus Segments English tokens Spanish tokens

CCMatrix 409.1 M 7,657.6 M 8,363.2 M
Paracrawl 269.4 M 4,374.1 M 4,762.7 M
OpenSubtitles 61.4 M 392 M 364.3 M
Europat 51.4 M 1,462.7 M 1,630 M
UNPC 25.2 M 4,374.1 M 4,762.7 M
CCaligned 15.3 M 161.4 M 172.9 M
MultiUN 11.4 M 285.1 M 335.9 M
Eubookshop 5.2 M 120.9 M 136.8 M
DGT 5.1 M 87.2 M 99.8 M
4 https://opus.nlpl.eu/.
5 https://commoncrawl.org/.
130 Antoni Oliver
browses the World Wide Web and stores the text of the webpages and
archives. This organization provides this data for free, and a new crawl is
completed approximately every month by randomly exploring URLs. In
this way, terabytes of web pages in various languages are obtained each
month. Each of these crawls is a snapshot of what is available at that
moment in time. Using different snapshots, a set of pre-processing steps are
performed: elimination of repeated paragraphs, automatic determination
of the language in which the web page is written, and filtering of low-qual-
ity content. Once the snapshots are pre-processed, a multilingual sentence
embedding space is learned. In this space, sentences (in their vectorized
forms in the multilingual embedding space, not in their original form con-
taining the actual source-language words) that are semantically similar
occur close to one another, regardless of the language they are written in.
To determine the extent to which two sentences in different languages are
mutual translations or not, we can calculate the distance between their
vector representations; an English sentence like, ‘Long live the King!’ and a
French sentence, ‘Vive le roi!’ should be very close together in the multilin-
gual embedding space, such that without any further knowledge or com-
plex calculation, we can assume them to be translations of one another.6
In this way, a large amount of parallel data for many language pairs is
created. Each parallel corpus provides a score for each segment pair indicating
the confidence of the segments being translation equivalents, which can
be used if required to prune low-scoring pairs. Schwenk et al. (2021) also
demonstrate that NMT systems trained with CCMatrix for several language
pairs performed better than systems trained with other popular corpora.
6.4.3 Corpus Combination
To train our NMT system, we want to select a given number of parallel

segments from CCMatrix. The selected segments should be the most
similar segments to those we plan to translate so that the best possible
quality can ensue. In our experiments, we use the source segments of all the
books of the Sherlock Holmes series by Conan Doyle. We use the source
segments only, and all these works are in the public domain.
To select the segments, we use the script MTUOC-corpus-combination
.py from the MTUOC (MT at the Universitat Oberta de Catalunya)7 (Oliver,
2020). This script can combine a small monolingual or parallel corpus (cor-
pus A) and a very large parallel corpus (corpus B). It can select the most
similar segments to the source segments of corpus A from corpus B. To be
able to select the most similar segments to those in the source language of
6 One common measure used is the cosine distance.

7 https://github.com/aoliverg/MTUOC.
corpus B, we first need some tool capable of describing or characterizing the

language used in the corpus. This is achieved using a language model; to
decide whether a given sentence is similar to those in corpus B, we can per-
form the same kind of counts in this segment and compare this count to the
counts in the language model.8 The user can indicate the corpus to combine
and the number of parallel sentences to be selected.
In our experiments, we used a monolingual English corpus from all the
works of the Sherlock Holmes series except The Adventures of Sherlock
Holmes. We excluded this work because we will perform the evaluation
using 1,000 segments from this book, and using its contents to train the
system would distort the results. In Table 6.2 we can observe the works
we have used to create the monolingual corpus for selection and the
number of segments. Using this corpus, we created three corpora using the
combination program with different sizes: 1 million segments, 5 million
segments, and 10 million segments. We used these corpora to train three
different NMT systems, as explained in the next section.
6.4.4 Training of the NMT Systems
To train the systems, we used the Marian toolkit (Junczys-Dowmunt, 2018).

This toolkit provides programs to train the model and to start up a server
that provides the translations. The native Marian server is used along with
the MTUOC server, which takes charge of the pre-processing steps and
Table 6.2 Works used to create the monolingual English corpus for selection. The
numbers are for unique segments and tokens.
Work Segments Tokens

The Adventures of Sherlock Holmes 6,293 106,669
The Memoirs of Sherlock Holmes 6,152 100,505
The Return of Sherlock Holmes 7,815 113,712
His Last Bow 5,207 69,558
The Case-Book of Sherlock Holmes 6,740 84,952
A Study in Scarlet 3,216 47,635
The Sign of the Four 2,950 43,703
The Hound of the Baskervilles 4,128 63,396
The Valley of Fear 4,272 58,748
TOTAL (incl. The Adventures of Sherlock Holmes) 46,874 687,250
TOTAL (excl. The Adventures of Sherlock Holmes) 40,084 580,853
8 A measure used very often is perplexity, a measure that indicates how well a given model
(in our case the language model) predicts or describes a sample (in our case the given
sentence).
132 Antoni Oliver
communication with the client program. In order to extend the compatibil-

ity of the MT systems with a wide range of client programs, the MTUOC
server can use different communication protocols. It implements its own
protocol, but it can also simulate the protocols used by Moses, OpenNMT,
NMTWizard, and ModernMT. This feature allows the MTUOC server to
be used with popular CAT tools such as Trados Studio or OmegaT.
Before training, the training and validation corpora were pre-pro-
cessed with SentencePiece (Kudo & Richardson, 2018)—see Section I.2,
Introduction, this volume, for an explanation of why this is usually needed
in NMT system-building. In our experiments, we assume a vocabulary size
of 64,000 and a vocabulary threshold of 50, i.e. each word had to exist at
least 50 times in the training data.
To train the system we used a Transformer-based NMT system (Vaswani
et al., 2017; see also Section I.2, Introduction, this volume). We used the
early-stopping criterion to stop training if no improvements were seen in
terms of cross entropy (Cybenko et al., 1999) or BLEU (Papineni et al.,
2002) after ten validation steps. After training was completed, we obtained
three models: one corresponding to the best cross-entropy score, another
corresponding to the best BLEU score, and a third one combining both. The
final engine uses an ensemble (a combination) of these three models with
weights 0.3, 0.3, and 0.4, respectively, i.e. the best cross-entropy system and
the best BLEU systems are equally weighted, with a slight weighting in the
ensemble architecture towards the best combined metric system.
With this configuration, we trained a total of four NMT systems:
• CCMatrix-10M-eng-spa: trained using the 10M best segments from

CCMatrix according to the margin score provided by the CCMatrix
corpus. This system will allow us to assess whether the selection of
segments improves the quality of the trained system.
• CCMatrix-Sherlock-Holmes-1M-eng-spa: trained using 1 million
segments selected from the CCMatrix corpus.
6.4.5 Automatic Evaluation of the Trained Systems
To perform the automatic evaluation of the trained systems, we used the

program MTUOC-eval9 to calculate the following metrics: BLEU (Papineni
9 https://github.com/aoliverg/MTUOC-eval.
et al., 2002); NIST (Doddington, 2002); Word Error Rate (WER) (Nießen,
2000); Edit Distance, based on the Levenshtein distance (Levenshtein,
1965); and Translation Edit Rate (TER) (Snover et al., 2006). See Way
(2018) and Section I.2, Introduction, this volume, for an explanation of
human and automatic evaluation of MT system quality.
For the evaluation in this experiment, we used a set of 1,000 segments
from The Adventures of Sherlock Holmes and a public domain translation
into Spanish. All the trained systems as well as Google Translate and
DeepL were evaluated, meaning that all 1,000 sentences in the test set were
translated by each of the systems, and the outputs were compared against
the human reference translations for those sentences. The overall score is
an average over all 1,000 sentence translations.
In Table 6.3 we can observe the evaluation scores for the English–Spanish
MT system. For BLEU and NIST, higher scores indicate better quality,
whereas for WER and TER lower values are better quality. %EdDist,
indicates the percentage of edit operations (usually insertions, deletions,
and substitutions) needed to convert the raw MT output into the human-
produced reference translation, so again, lower values mean better quality.
The system CCMatrix Sherlock Holmes 10M obtains the best scores for
BLEU and %EdDist, whereas DeepL receives better scores for NIST, WER,
and TER. As a conclusion, we can confirm that the CC Matrix Sherlock
Holmes 10M obtains competitive scores, very close to those from the two
popular commercial systems. It is also worth mentioning that CC Matrix
Sherlock Holmes 10M outperforms the CCMatrix 10M system, indicating
that the corpus selection technique is very productive. Even those systems
trained with the selection of 5M segments achieve better results for all
metrics, and the system with the selection of 1M segments obtains better
values on all metrics except %EditDist.
Given its superior performance, in the remaining experiments in this
chapter, we will use the CCMatrix Sherlock Holmes 10M system, along
with Google Translate and DeepL.
Table 6.3 Evaluation figures for the English–Spanish MT systems (figures in bold

indicate the best value for this measure).
Google DeepL CCMatrix CCMatrix CCMatrix CCMatrix

Translate (10M) Sherlock Sherlock Sherlock
Holmes Holmes(5M) Holmes(10M)
(1M)
BLEU 0.315 0.335 0.240 0.296 0.329 0.371

NIST 7.318 7.58 6.166 6.429 6.995 7.524
WER 0.508 0.491 0.602 0.582 0.551 0.51
%EdDist 42.431 41.691 48.351 50.823 45.897 41.588
TER 0.481 0.464 0.569 0.554 0.512 0.475
134 Antoni Oliver
6.4.6 Evaluation of the Training Systems Using Post-Editing Effort
Post-editing is what a professional post-editor does to convert the raw

machine-translated text into a correct translation. To evaluate post-
editing effort, we used PosEdiOn (Oliver et al., 2020), a post-editing tool
developed to collect information on different implicit and explicit effort
indicators. The tool has an intuitive graphical interface where the user
can post-edit a project. All the actions performed by the user (keystrokes,
mouse actions, going to another application, and coming back to the tool)
are recorded in a database along with the time taken to perform the action.
This allows its companion program, the PosEdiOn analyzer, to calculate a
series of indicators on the post-editing task. The user can pause the task at
any time to avoid errors in counting of time spent post-editing. That said,
as it is always possible for the user to forget to pause the post-editing task,
the program can calculate pruned values of all the measures, i.e. to avoid
including those segments with a post-editing normalized time (per token or
character) which is abnormally long.10
The PosEdiOn analyzer can calculate a long series of measures, but in
this study, we will concentrate on the following measures, all calculated
with the pruned values:
• Normalized time per token.

• Human variants of TER, BLEU, NIST, and Edit Distance, called HTER
(Snover et al., 2006), HBLEU, HNIST, and HEd, calculated comparing
the raw MT output with the post-edited segment.
• A series of measures proposed by Barrachina et al. (2009), namely: key-
stroke ratio (KSR—number of keystrokes divided by the total number
of characters in the raw machine-translated text); mouse-action ratio
(MAR—number of pointer movements plus one more counts per sen-
tence, divided by the total number of characters in the reference transla-
tion), and keystroke and mouse-action ratio (KSMR—KSR plus MAR).
To perform this evaluation, we chose Adventure V. The Boscombe

Valley Mystery, consisting of 633 segments. We split this adventure into
12 fragments, 11 of them having 50 segments, and the last one 83. We
translated these fragments into Spanish with our trained Marian MT
system (fragments 1, 4, 7, and 10), Google Translate (fragments 2, 5, 8,
and 11), and DeepL (fragments 3, 6, 9, and 12).
The post-editing task was proposed to master’s students in Translation
and Technologies at Universitat Oberta de Catalunya (UOC). 12 students
accepted the task, but only six of them completed all the post-editing tasks,
10 Longer than the mean time plus two times the standard deviation.
so we only consider the post-edits of these six students in what follows.

The students were asked to complete the tasks in order, and they did not
know which MT system was used to translate each fragment. All the stu-
dents used PosEdiOn to perform the task. After the task was completed,
we used the PosEdiOn analyzer to calculate the above-mentioned pruned
indicators. In Table 6.4 we can observe the obtained values. The analysis
of the values of the different metrics obtained for the post-editing tasks fol-
lows the same tendency as the results obtained with the automatic evalu-
ation metrics, but now even the HBLEU metric is worse for our system.
Regarding the normalized time (time in seconds spent to post-edit each
token), the best (lowest) value is achieved by Google Translate, but values
for normalized keystrokes (keys pressed to post-edit each token) and nor-
malized mouse actions (use of the mouse to post-edit each token plus one
for each segment) are better (lower) for DeepL. All the remaining scores
(HBLEU, HNIST, HTER, and HEd) are also better for DeepL, leading to
the conclusion that for the post-editing task, the best system is DeepL, and
that our system shows much worse values for all metrics.
6.4.7 Creation of Bilingual E-books
All the scripts necessary to create the bilingual e-books using an MT system
are publicly available on GitHub.11 The full process can be divided into the
following steps:
Table 6.4 Figures from the analysis of the post-editing tasks (figures in bold indi-
cate the best value for this measure).
Marian Google Translate DeepL

Norm. time (s.) 0.148 0.076 0.081
Norm. keystrokes 1.908 1.287 1.108
Norm. mouse. act. 0.310 0.270 0.246
KSR 0.430 0.261 0.224
MAR 0.070 0.055 0.050
KSRM 0.500 0.316 0.274
HBLEU 0.617 0.756 0.779
HNIST 7.098 8.166 8.326
HTER 0.272 0.158 0.143
HWER 0.320 0.187 0.170
HEd 21.948 12.963 11.624
11 https://github.com/aoliverg/InLector.
136 Antoni Oliver
• Obtaining the source text: the main source for our books is Project
Gutenberg.12 As a source file, we can use two different formats: EPUB
or plain text.
• Conversion of the source text to DocBook, a standard XML file format
for books and other documents.
• Creation of the bilingual DocBook: using the MT system, we create a
DocBook file where each source segment is linked to the corresponding
machine-translated target segment.
• Creation of the bilingual e-book in the final format: we create an EPUB
and an HTML file using the provided scripts or other standard tools for
handling DocBook files.
6.4.8. Evaluation of the Reading Experience of the Bilingual E-books
To assess the reading experience and the user’s opinions of the bilingual
e-books, we distributed a questionnaire developed in Google Forms with
12 questions. The questionnaire was distributed via a mailing list for bach-
elor and master students of translation at UOC. This mailing list is also
open to former students and to any person interested in translation. No
personal contact data was collected from the participants. The users were
asked to read one of the adventures in The Adventures of Sherlock Holmes
(whichever they wanted), in the English–Spanish bilingual version, and to
answer the questions after reading the adventure.
We received a total of eight answers to the questionnaire. Here we
present a summary of the results.
Question 1. In which format did you read the adventure? Half of the
users read it in HTML and the other half in EPUB.
Question 2. On which device did you read the adventure? Most of the
users (seven out of eight) used a computer and one used a
tablet.
Question 3. Which adventure did you read? Here we saw a great
disparity:
• Two read V. The Five Orange Pips.
• Two read IV. The Boscombe Valley.
• One read II. The Red-Headed League.
• One read VI. The Man with the Twisted Lip.
• One read VIII. The Adventure of the Speckled Band.
• One read XI. The Adventure of the Beryl Coronet.
12 www.gutenberg.org/.
Question 4. Which is your level of English? One reported a B2, Two

C1s, and five C2s. These language levels are defined by
the Council of Europe’s Common European Framework
of Reference (CEFR).13
Question 5. Do you have a certificate of your level of English? 50%
answered yes, and 50% answered no.
Question 6. Have you read this adventure before? Three respondents
had read it before, and five had not.
Question 7. How many times have you consulted the translated ver-
sion? Seven of the readers consulted the translation five or
fewer times, and one more than five times.
Question 8. Did the published translation help you to understand the
fragment you consulted? The reader had to choose on a
scale from 1 (helped very little) to 5 (helped a lot). Five
respondents answered 4 or 5, two answered 3, and one
answered 1. The average score was 3.75.
Question 9. Is the quality of the published translation good enough for
you to understand the content? The reader had to choose
on a scale from 1 (very low quality) to 5 (very high qual-
ity). Six readers answered 4 or 5 and two answered 1 or 2.
The average score was 3.88.
Question 10. Would you like to read more books in this bilingual elec-
tronic format? The reader had to choose on a scale from 1
(not at all) to 5 (very much). Seven respondents scored 4
and 5, and one scored 2. The average score was 4.5.
Question 11. Would you recommend to your friends reading books in
this bilingual electronic format? The reader had to choose
on a scale from 1 (not at all) to 5 (very much). All the
respondents score 4 and 5. The average score was 4.88.
Question 12. Do you think this kind of bilingual e-book can promote
reading novels in the original version? The reader had to
choose on a scale from 1 (not at all) to 5 (very much). All
the respondents scored 4 and 5. The average score was
4.88.
6.5 Discussion
In this research, we trained an NMT system tailored for translating literary
works for a given author using only the original works in the public
domain and a freely available very large parallel corpus. A small subset of
the human translation in the public domain of the chosen work was used
13 www.coe.int/en/web/common-european-framework-reference-languages.
138 Antoni Oliver
to evaluate the trained system along with two popular commercial NMT
systems, Google Translate and DeepL. The NMT system was trained
with the goal of producing a bilingual e-book, useful for learners of the
source language to read the novel in the original version and to consult the
translation of difficult sentences.
As expected, most of the metrics used for evaluation are worse for our
trained system. However, is it still a useful system for the production of
bilingual e-books? The evaluation of the reading experience shows that
the trained system produces useful translations, as 62.5 % of the read-
ers thought that the published translations helped them understand dif-
ficult sentences, and 75% thought that the translation was good enough to
understand the content. Most of the readers (87.5 %) would like to read
more books in this bilingual electronic format, and 100% would recom-
mend this kind of e-book to their friends. It is also worth noting that 100%
of the respondents think that these bilingual e-books can promote reading
the original version of novels.
6.6. Conclusions and Future Work

NMT systems achieve very high quality, but as recently stated by
Guerberof and Toral (2022), quality levels are not yet high enough for
literary translation when the goal is to produce translations to be read as a
final product. Despite that, can NMT systems be useful for literary works
for different purposes, such as producing bilingual e-books for language
learners?
In this chapter, we trained an NMT system for English–Spanish
tailored to translate literary works by a given author (more precisely, a
given series of books by a given author). To train the system, we did not
use human translations under copyright, but only source texts already
in the public domain. These source texts are used to automatically select
suitable parallel segments from a very large freely available parallel corpus.
Compared with Google Translate and DeepL, the trained system achieves
much worse automatic evaluation scores, but a reading experiment shows
that it produces useful translations for the creation of bilingual e-books.
As an avenue of future work, we plan to improve the quality of the
MT system, experimenting with other NMT architectures and training
processes. We also plan to use other large parallel corpora from which
to select the segments and compare the automatic evaluation metrics
obtained. We also plan to extend the experiments to further language pairs,
starting with English–Catalan, French–Spanish, and French–Catalan. With
all these systems, we will create a larger catalog of bilingual e-books to
engage more readers to participate in the reading experiments.
References
Barrachina S., Bender, O., Casacuberta, F., Civera, J., Cubel, E., Khadivi, S.,
Lagarda, A., Ney, H., Tomás, J., Vidal, E., & Vilar, J.M. (2009). Statistical
approaches to computer-assisted translation. Computational Linguistics 35 (1),
3–28. https://doi.org/10.1162/coli.2008.07-055-R2-06-29
Cybenko, G., O’Leary, D.P., & Rissanen, J. (Eds.). (1999). The mathematics of
information coding, extraction and distribution. Springer. https://link.springer
.com/book/10.1007/978-1-4612-1524-0
Doddington, G. (2002). Automatic evaluation of machine translation quality using
n-gram co-occurrence statistics. In Proceedings of the Second Conference on
Human Language Technology (HLT 2002) (pp. 128–132). https://aclanthology
.org/www.mt-archive.info/HLT-2002-Doddington.pdf
Guerberof, A., & Toral, A. (2022). Creativity in translation: Machine translation
as a constraint for literary texts. Translation Spaces, 11, 184–212. https://doi
.org/10.1075/ts.21025.gue
Hadley, J.L. (2020). Literary machine translation: Are the computers coming
for our jobs? Counterpoint, 4, 14–18. https://www.academia.edu/44701250/
Literary_machine_translation_Are_the_computers_coming_for_our_jobs
116–121). Association for Computational Linguistics. https://aclanthology.org
/P18-4020/
Kudo, T. & Richarsdon, J. (2018). SentencePiece: A simple and language
independent subword tokenizer and detokenizer for Neural Text Processing.
XVII: The Qualities of Literary Machine Translation (pp. 1–9). European
Association for Machine Translation. https://aclanthology.org/W19-7301.pdf
Levenshtein, V.I. (1965). Binary codes capable of correcting deletions, insertions,
and reversals. Soviet Physics Doklady, 10(8), 707–710. https://nymity.ch/
sybilhunting/pdf/Levenshtein1966a.pdf
Association for Machine Translation. https://aclanthology.org/W19-7302.pdf
Moorkens, J., & Lewis, D. (2019). Research questions and a proposal for the
future governance of translation data. JosTrans, 32, 2–25. https://www.jostrans
.org/issue32/art_moorkens.pdf
Nießen, S., Och, F., Leusch, G., & Ney, H. (2000). An evaluation tool for machine
translation: Fast evaluation for MT research. In Proceedings of the Second
International Conference on Language Resources and Evaluation (LREC’00)
(7p.). European Language Resources Association (ELRA). https://aclanthology
.org/L00-1210/
140 Antoni Oliver
Oliver A., Toral, A., & Guerberof, A. (2019). InLéctor neural machine translation
for the creation of bilingual ebooks. In Proceedings of Machine Translation
Summit XVII: The Qualities of Literary Machine Translation, Dublin, Ireland
(p. vii). European Association for Machine Translation.
Oliver, A. (2020). MTUOC: Easy and free integration of NMT systems in
professional translation environments. In Proceedings of the 22nd Annual
Conference of the European Association for Machine Translation (pp. 467–
468). https://aclanthology.org/2020.eamt-1.55
Oliver, A., Alvarez, S., & Badia, T. (2020). PosEdiOn: Post-editing assessment
in PythOn. In Proceedings of the 22nd Annual Conference of the European
Association for Machine Translation (pp. 403–410). https://aclanthology.org
/2020.eamt-1.43
Oliver, A., Toral, A., & Guerberof-Arenas, A. (2022). Bilingual e-books via
neural machine translation and their reception. In J.L. Hadley, K. Taivalkoski-
Shilov, C.S.C. Teixeira, & A. Toral (Eds.), Using technologies for creative-text
translation (pp. 89–115). Routledge. https://doi.org/10.4324/9781003094159
meeting of the Association for Computational Linguistics, Philadelphia, PA (pp.
311–318). Association for Computational Linguistics. https://aclanthology.org
/P02-1040/
Schwenk, H., Wenzek, G., Edunov, S., Grave, E., & Joulin, A. (2021). CCMatrix:
Mining billions of high-quality parallel sentences on the web. In Proceedings
of the 59th Annual Meeting of the Association for Computational Linguistics
and the 11th International Joint Conference on Natural Language Processing
(Volume 1: Long Papers) (pp. 6490–6500). Association for Computational
Linguistics. https://aclanthology.org/2021.acl-long.507
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., & Makhoul, J. (2006). A study
of translation edit rate with targeted human annotation. In Proceedings of the
7th Conference of the Association for Machine Translation in the Americas:
Technical Papers (pp. 223–231). Association for Machine Translation in the
Americas. https://aclanthology.org/2006.amta-papers.25/
using SMT. In Proceedings of the Fourth Workshop on Computational
Linguistics for Literature, NAACL, Denver, CO (pp. 123–32). https://
aclanthology.org/W15-0714.pdf
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.L., Gomez, A.N.,
Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In Advances
in Neural Information Processing Systems 30: Annual Conference on Neural
Information Processing Systems 2017, Long Beach, CA (pp. 5998–6008).

.1803.08409
7 Machine Translation of Chinese
Fantasy (Xianxia) Novels
An Investigation Into the Leading Websites
Translating Chinese Internet Literature Into
English
Shuyin Zhang
7.1 Introduction
It is often said that translating literary texts is ‘the last bastion of human
translation’ (Toral & Way, 2014, p. 174) in the competition between
human and machine modes of translation. This is evident in the following
statement:
Computer programs are producing translations—not perfect

translations, for that is an ideal to which no human translator can
aspire; nor translations of literary texts, for the subtleties and nuances
of poetry are beyond computational analysis; but translations of
technical manuals, scientific documents … medical reports.
(Hutchins, 1995, p. 431)
Machine translation (MT) of literary texts has long been seen as

controversial. In the past, it has been used predominantly for technical
and informative texts rather than literary texts. Novels are, of course,
not manuals, which leads to the question of how computers can possibly
rival humans in dealing with creative and literary works. The advent of
electronic literature (Hayles, 2008), or digital literature, has in fact turned
a new page. Like many other cultural products, literature that is influenced
by the booming internet industry and the growing popularity of mass
media has gone through a radical revolution and is becoming increasingly
digitalised and commercialised. Various new forms and genres of writing,
such as combinatory poetics, hypertext fiction and interactive fiction—to
name but a few—have come into being (Rettberg, 2019, p. 12). Created
primarily or exclusively on and for the internet, in many ways, electronic
literature has added to traditional, printed forms of literature by widening
the readership and contributing additional forms to its repertoire, and is
facilitating intercultural communication through translation at the same
time as refreshing the outdated and ‘biased’ perception of MT.
DOI: 10.4324/9781003357391-10
Machine Translation of Xianxia Novels 143
Chinese internet/net literature (wangluo wenxue), also known as

Chinese online literature, is a form of electronic literature ‘created by
netizens and published on the Internet, which is meant to entertain internet
users or engage them’ (Ouyang, 2008, p. 4, in my translation), came into
being 30 years ago with the advancement of modern computers and digital
network technology. A more recent definition refers to it as ‘the original
Chinese-language literary writings produced by Chinese online writers for
publication in commercial or private internet cyberspaces, which are read,
appreciated and responded to by online readers via internet’ (Duan, 2018,
p. 669). Over the past two decades, dozens of Chinese internet literature
platforms have sprung up, including Qidian,1 Chuangshi,2 Zongheng3 and
Jinjiang,4 among others, which has unexpectedly sparked the international
dissemination of Chinese culture and literature. Since the 1980s, there has
been a call for Chinese literature, particularly classical literature, to ‘go
abroad’. However, the task has been a hard one. Despite years of effort,
even the books brought out by the most prominent reading project, the
Panda Books Translation Series,5 were barely read outside China (Wang
& Xie, 2013). In contrast, Chinese popular culture, in particular Chinese
net literature, with its characteristics of popularisation, interactivity
and intermediality6 has attracted a substantial number of readers across
different cultures during the last few years. By 2019, 10,000 Chinese
online literary titles had been read by readers from more than 40 countries
and regions (China Literature, 2020, as cited in Jia, 2020). In 2019, the
overseas market reached 460 million yuan, with the number of overseas
readers reaching 31.94 million, a figure that was expected to increase
to 50 million in 2021 (iResearch, 2020). Its current overseas market is
promising and expects further growth.
Each Chinese internet literature platform has its own taxonomy of liter-
ary genres, but by and large, the core genres cover fantasy novels, Wuxia
novels, Xianxia novels, military novels, city/modern novels and historical
novels, with the boundaries between them sometimes being indistinct. The
subject of this chapter, Xianxia (仙xian: immortal; 侠xia: chivalrous) nov-
els, which originated in the Tang dynasty, had once prospered during the
Qing dynasty, and have since been revived as a sub-genre of net literature,
keeping pace with the upsurge of online media and virtual communities in
1 www.qidian.com/.
2 https://chuangshi.qq.com/.
3 www.zongheng.com/.
4 www.jjwxc.net/.
5 This project was initiated in 1981 by Chinese Literature Press, presenting the literary face
of China to the world.
6 This term refers to ‘interconnectedness of modern media of communication’ (Jensen, 2016,
p. 1).
144 Shuyin Zhang
the early 2000s. Heavily inspired by Daoism, Buddhism, ancient Chinese

mythology and classical Chinese literature, this genre portrays an illusory
and ethereal world of gods, immortals, humans, demons, devils and ghosts
that is distinct from Western fantasy novels whose key elements are magic,
wizards, vampires, fire-breathing dragons, knights and Gothic Castles.
Most Xianxia novels have a relatively fixed pattern: ‘protagonists (usually)
attempt to cultivate immortality, seeking eternal life and the pinnacle of
strength’.7 The story is usually set in a legendary era, either a golden age
with thousands of talents coming together or a dark age under the reign
of evil forces. A hero with magic powers, inherent or gained, born as a
nobody (though it sometimes turns out that the hero is actually of noble
birth), becomes an immortal or a mighty figure who dares to challenge the
hierarchy.
As a web-based genre, Xianxia literature tends to focus less on literary
qualities. In other words, whatever is valued and appreciated by traditional
literature is reduced to a significant extent in cyberspace, namely the artistic
use of language, vivid descriptions and carefully devised plots. Nevertheless,
by incorporating elements from video games, comics and science fiction,
Xianxia literature has gained popularity worldwide. Its popularised,
entertainment-oriented and down-to-earth content distinguishes Xianxia
novels from serious literature, reducing translation difficulties, and thanks
to burgeoning translation technologies and online platforms that translate
Xianxia literature, it is now becoming more accessible to foreign readers.
The major websites, either set up by large business companies or evolving
by way of fan communities, are Webnovel,8 Wuxiaworld9 and LNMTL
(light novel machine translation)10 among others, offering novels translated
by humans and machines and by fans and professionals.
Unlike the mainstream myth that considers that MT is doomed to failure
in literary contexts, this chapter represents an initial attempt to look at
the other side of the coin by arguing that Chinese Xianxia literature, one
of the most popular genres of Chinese internet literature among overseas
readers, is suitable for MT. It aims to pinpoint the characteristics of
Xianxia literature and to analyse the advantages of applying MT against
a backdrop of the thriving overseas market for Chinese net literature. The
prospects for the MT of Xianxia literature are showcased by assessing
the translation modes and methods adopted by leading platforms in the
industry.
7 www.wuxiaworld.com/page/general-glossary-of-terms.
8 Webnovel (www.webnovel.com) is owned by the Tencent-backed digital content company
China Literature Limited.
9 www.wuxiaworld.com
10 lnmtl.com
7.2 Research on Literary Machine Translation

MT has caught the attention of Translation Studies scholars since the
2000s. The key issues under investigation have included translation qual-
ity, translator/posteditor productivity, evaluation of MT, and cogni-
tive effort and economic cost in machine-translated texts with different
combinations of human translation (HT), raw MT and MT with post-
editing (MTPE) (Kenny, 2019). However, many scholars perceive literary
translation as a traditional and age-old practice that neither requires nor
desires any technological assistance beyond the standard functionalities
of word processing and internet search, as mentioned in the Introduction
to this book, and fear the potential side effects of computer technolo-
gies. Thus, until recently, not much research was done on MT for liter-
ary texts, but Voigt and Jurafsky (2012), Jones and Irvine (2013) and the
studies reviewed in the Introduction of this volume discuss the difficulties
involved and improvements in literary MT. Recent literature reflects on
ethical issues associated with literary MT in broader cultural debates, as
in Taivalkoski-Shilov (2018) and Kenny and Winters (2020). While most
texts examined are formal literature—novels by Albert Camus for exam-
ple—and language pairs consist of languages that are not too distant from
one another, such as English and French, English and Spanish and Spanish
and Catalan, studies on the translation of popular literature between unre-
lated languages like English and Chinese are rare. Indeed, research into
MT between Chinese and English has largely been engaged with non-liter-
ary discourses. Most studies examine translations from English to Chinese
rather than the other way around. A large body of research argues that
machines cannot excel at literary translation, in particular when dealing
with metaphors, implicatures, words from another register or a distant
culture, aesthetic elements in the description of characters and scenery and
emotional expressions (see, for example, Wu, Zhang, & Zhu, 2018). Very
few enquire into the possibilities of literary MT and simply compare the
HT and MT of a particular novel, usually a classic (see, for example, Jiang
& Yu, 2017), thus reflecting only a limited perspective. Shih (2016), who
scrutinises the web audience’s preference for different machine translations
of online Chinese folktales, is an exception. The nature of Shih’s work,
which focuses more on the reader’s acceptance, is very different from the
present chapter, attempting to explore the role of MT in fan translation of
Xianxia literature, however.
The volume of research on Chinese internet literature is small, with most
springing from Communication Studies rather than Translation Studies.
For instance, Shao, Ji and Xiao (2018) review the online platforms trans-
lating Chinese web novels into English in detail, but their emphasis is on
the international distribution of Chinese internet literature. Li (2019) also
analyses this overseas distribution, but from a sociological perspective—the
146 Shuyin Zhang
actor-network theory, which is based only on Wuxiaworld—to study

modes of communication. Existing literature mainly discusses Chinese
internet literature in general, whereas Xianxia literature has seldom been
touched upon, especially in Translation Studies. Fewer than a dozen
papers have been published within the past five years, mainly discussing
the translation of culture-specific words (see, for example, Zhu, 2020)
and the construction of the image of China in translations (see, for exam-
ple, Yang, 2019). It is rare to see studies contextualising Xianxia novels
within the background of literary MT, and if in fact there are any, the use
of MT is addressed only in passing. For example, Ruan (2019) discusses
the overseas dissemination of Chinese web novels and its implications for
Translation Studies, where MT is only one of the many aspects involved.
Innovative as these studies are, they do not examine the feasibility of using
MT to fill the gap between the progressively expanding overseas market
for Xianxia literature and the resultant current translation activity, nor are
the various websites translating Xianxia literature discussed. Although it
is constantly being challenged, MT deserves an in-depth exploration of its
potential with regard to literary translation.
7.3 The Translation of Xianxia Novels: An Examination of the

Leading Websites
In traditional literary translation contexts, readers are not usually immedi-
ately and directly involved in the translation process. In contrast, in the age
of Web 2.0, for internet literature translation (particularly the translation
of Xianxia literature) readers are highly engaged in their close interaction
with translators and texts and, as such, assume a position of power. They
not only comment on the translations instantly, influencing subsequent
translations, but also edit available MT output in a new translation envi-
ronment and even carry out the translations (including using MT) them-
selves, either individually or as a group, switching their role from reader
to translator. This new environment has fueled the growth of fan-based
translation communities.
Two pioneer online fan-based communities are of great significance
in terms of the introduction of Xianxia literature to the world. The first
is Wuxiaworld, established in North America in 2014 by the Chinese-
American Lai Jingping, also known as RWX. Lai, a fan of Xianxia novels,
created Wuxiaworld and built up his translation team after successfully
translating the popular novel Coiling Dragon on his own. Wuxiaworld was
once the most influential fan-based website that attracted more than 30
million readers from over 100 countries. The other one, Gravity Tales, was
founded in 2015 by Kong Xuesong, a fan of Wuxiaworld. In addition to the
English translations of Chinese Xianxia novels, Gravity Tales also provides
original Xianxia novels written in English, such as Tina Lynge’s Blue
Phoenix. The translation model for these early platforms is fan translation
by human translators, a pull model where ‘end-users increasingly demand
translation or do it themselves’ (Drugan, 2013, p. 159). It is worth noting
that both websites were acquired by larger companies that own the rights
to the works in the last two years.
Commercial platforms with the copyright to the novels to produce offi-
cial translations adopt a push model where established clients and produc-
ers decide what to translate and deliver to consumers (Drugan, 2013, p.
159), offering professional translation by human translators. Webnovel,
the international version of Qidian, which translates Chinese novels from
Qidian, is a prime example. Founded by China Literature Limited in 2017,
Webnovel, with more than 375,000 original works, is one of the largest
internet literature websites. Unlike its predecessors, Webnovel offers not
only Xianxia but many other genres and content in six languages. Today,
the number of profit-oriented platforms is increasing, and their competi-
tors—the fan-based online translation communities—have undergone
a transformation and become more professional, and some have estab-
lished a collaboration or merged with official websites. However, a gap
still exists between the large-scale overseas propagation of Chinese net
literature and the current capacity for HT. The speed of HT varies widely
and is generally too slow to cater to the readers’ curiosity concerning
updates. Besides, some novels have not yet been selected for translation
by human translators and commercial websites, all of which has fueled
the need for MT.
Many fans have thus started to produce MT output themselves, mainly
using freely available platforms, which is also a case of a pull model. This
model that relies on MT (usually assisted by human pre-and post-editing)
rather than HT has two subtypes, a collective one involving more than one
translator, and an individual one. In terms of the collective model, LNMTL
is one of the earliest Xianxia literature translation platforms to adopt this
translation method. Compared with Wuxiaworld, LNMTL is smaller, less
organised and ‘wilder’, offering only machine-translated Xianxia novels,
but it is popular despite its dubious legal status. Founded in 2016 by an
engineer with no knowledge of Chinese, LNMTL’s purpose is to ‘make it
easy to read machine translation of Chinese web literature’.11 According to
the discussion on the forum on Novel Updates,12 it is believed that LNMTL
uses Systran to translate the novels although no explicit statement can be
found on LNMTL’s website.13 When reading content on LNMTL, readers
11 https://lnmtl.com/about.
12 Novel Updates (https://forum.novelupdates.com) is a directory of Asian translated novels
and a platform for users who enjoy reading web novels.
13 https://forum.novelupdates.com/threads/how-does-lnmtl-translate-their-chapters.19240/.
148 Shuyin Zhang
Figure 7.1 S creenshot of LNMTL’s translation of Volume 45, Book 17, Desolate
Era (tonally inverted). https://lnmtl.com/chapter/desolate-era-book-45
-chapter-17.
can spot terms highlighted, and by clicking on them, the original texts
in Chinese will appear which allows the reader to look up the terms in
the dictionary more efficiently, thus smoothing the reading experience, as
shown in Figure 7.1.
In addition, readers are involved in the translation process by freely
communicating with one another under the translation posts and are
encouraged to polish the MT output. LNMTL’s post-editing process is
open to all its readers. Specifically, they can add glossary terms, modify
phrases, adjust sentence structures and rewrite sentences by creating new
propositions. Once the number of changes reaches 200, retranslation will
be triggered. LNMTL also urges its readers to ‘vote on which series to pick
up’ and ‘message about the blunders/missing chapters’,14 giving its readers
some of the same rights as editors and translators.
LNMTL’s model has been widely acknowledged by its readers judg-
ing by the feedback left on the website. There have, however, been com-
plaints about the translation quality. LNMTL has defended itself by
14 https://lnmtl.com/about.
Machine Translation of Xianxia Novels
149
Figure 7.2 Screenshot of the introductory page of Desolate Era (tonally inverted). https://lnmtl.com/novel/desolate-era.
150 Shuyin Zhang
replying, ‘Machine translators don’t make mistakes randomly, which cre-

ates patterns. Brains can perceive patterns and fix them subconsciously’.15
Surprisingly, LNMTL does not boycott its rival Wuxiaworld’s translations.
On the contrary, next to its translations, there is usually a sign announcing
that the translations are produced by MT and ‘for human translations visit
Wuxiaworld’, as shown in Figure 7.2.16
The other sub-type refers to MT output generated by individual read-
ers as independent user translators. Some are dilettante ‘MTLers’ who
do not know Chinese, but this does not stop them from translating using
machines. In this case, MT empowers monolingual readers to assume the
role of an editor and translator. If a novel’s official translation and fan
translation are not updated in time, eager readers will machine-translate
one or more chapters either with or without pre- and post-editing, which
also explains the emergence of websites like LNMTL. According to the
discussion on the forum on Novel Updates, user translators use various
translation engines such as Google Translate, Baidu Translate, Yandex
and Papago, among others, either with or without pre- and post-editing.
The use of CAT tools is not a popular option at the moment partly because
they are not professional translators and using CAT tools is not cost-effec-
tive. Nonetheless, the free translation memory tool, OmegaT, has been rec-
ommended by some user translators.17 Unlike LNMTL, which translates
on regular occasions, most solo MTLers translate only sporadically and
keep the translations only to themselves. However, there are people who
have formed a habit of machine-translating novels and posting the MT
output on Twitter and in blogs and asking for donations.
Many readers, including translation users, hold a favourable view of
machine-translated novels, especially collective MT output, namely works
on LNMTL, judging by positive feedback on the forum on Novel Updates.
The LNMTL glossary is what they praised the most. For instance,
Roxxman, a member of Novel Updates, comments on the advantages of
reading an LNMTL novel, noting that LNMTL ‘uses its own user-edited
glossary for each separate novel after its first MTL (machine translation).
The glossary is also continually renewed if enough people are willing to
keep updating it’.18 LNMTL’s glossary is collective and open to all and
is, therefore, more comprehensive and accurate than works by individuals
who cannot read Chinese but still prefer to machine-translate themselves.
Such feedback justifies the fact that despite the somewhat erratic quality
seen, many readers prefer to stick to reading MT output on LNMTL. In
15 https://lnmtl.com/faq.
16 https://lnmtl.com/novel/desolate-era.
17 https://forum.novelupdates.com/threads/so-how-do-you-translate.66056/.
18 https://forum.novelupdates.com/threads/best-mtl.120521/#post-6157669.
fact, assisted with human post-editing, the translation of terminology on

LNMTL does not differ very much from the translation on Wuxiaworld,
especially for proper nouns. For terms that are different, Wuxiaworld’s
translations are more refined and accurate than LNMTL’s. We now pro-
vide a brief example, where the phrase ‘青袍牛角人完全明白纪宁的心情’
(Vol.45, Ch.17)19 from the phenomenal novel Desolate Era is selected.
LNMTL translates the phrase to ‘green gown cow-horned man under-
stands completely the Ji Ning mood’,20 and Wuxiaworld translates it to ‘the
horned, azure-robed man completely understood how Ji Ning was feeling
right now’.21 Two terms are identified and underlined in the source text.
MT and HT translate the character’s name the same but differ in the other
term, ‘青袍牛角人’, which contains a colour word. Wuxiaworld’s version
is more refined than LNMTL’s. Despite such imperfections, LNMTL pro-
vides a straightforward translation of the terms. Arguably, as long as the
translation of terminology—the kernel of the genre—is correct, readers
tend to be willing to sacrifice the overall flow and bear with the grammati-
cal inaccuracies and ambiguous sentences to save on subscription and cut
down on waiting time.
There are, however, more sceptical views to be seen. Some claim that
certain Xianxia novels are unsuitable for MT, especially those with an
excessive number of idioms or which use a convoluted style of writing. For
example, Alvastar, a member of Novel Updates, notes that ‘some novels
are amazingly easy to read [in] MTL, even with highly specialized terms.
While [sic] other novels require me to stop…with no specialized terms,
simply because the author writes in a convoluted way’.22 Indeed, certain
features might increase translation difficulties and selecting the criteria
required by Xianxia novels for MT might be a possible topic concerning
the improvement of MT in future research.
Some objections are even stronger, with MT perceived as a threat
that weakens the status of professional translators. An MTLer named
dimsumdimsum, who posted her machine-translated novel in her blog and
asked for donations, received some hostile tweets such as the following:
‘amazed at how people think translating is putting text into a machine
and letting it do the job for you, then editing the text based on your
understanding of the language used for translation’ and ‘thinking that
it’s actually worthy of donations...’.23 She raised a discussion about these
tweets on Novel Updates, hoping to win support so that she could continue
19 https://vipreader.qidian.com/chapter/2502372/82320305/.
20 https://lnmtl.com/chapter/desolate-era-book-45-chapter-17.
21 www.wuxiaworld.com/novel/desolate-era/de-book-45-chapter-17.
22 https://forum.novelupdates.com/threads/best-mtl.120521/#post-6157669.
23 https://forum.novelupdates.com/threads/opinions-on-mtlers.117455/
152 Shuyin Zhang
using MT as an amateur translator. However, the replies to her post were

harsh as well, for example, a member called jjbookworm said:
Translation takes a lot of time and effort. … many readers prefer

quantity over quality, and even when they know the translation is
obviously better though slower, many still choose to read the MTL. …
it feels like translators can be so easily replaced, and that their work
was for nothing.
Indeed, MT lowers the threshold to access translation, attracting more

reader-translators and facilitating intercultural communication. Chinese
internet literature is admittedly promoted by MT, at least to some degree.
Nevertheless, MT still poses a threat to human translators in the eyes of
many people and will undoubtedly be confronted with numerous obsta-
cles before being acknowledged, especially within the field of literary
translation.
7.4 The Opportunity for MT in Literary Texts: The

Commercialisation of Internet Literature and the Intrinsic
Features of Xianxia Novels
The intercultural communication surrounding Xianxia novels is rooted
in fan-based communities where MTPE is a popular translation mode.
The fan translation of Xianxia literature, as a type of user-generated
translation (O’Hagan, 2009), relies heavily upon the expansion of com-
munity translation and crowdsourcing translation, thus evolving into
a more cooperative, web-based, technology-driven mode of transla-
tion. Like other user-generated translations (fansubbing24 for example),
copyright licensing is a critical issue. In addition, the lack of translation
quality and inconsistent translation frequency may upset readers. Unlike
fansubbing, which mainly involves HT, MT plays an important part in
the fan translation of Xianxia novels. Compared with novels translated
by human translators, such as those published on Wuxiaworld, which
use more natural language and have a feeling of literariness, the read-
ing of MTs output may not be particularly smooth. However, is it a
more refined version that the readers are longing for as opposed to an
instant translation that gets the storyline right after a new episode is
released? What is it that matters to the readers of Xianxia literature: the
speed of translation, the story itself or the language? Despite such con-
troversies and limitations, websites like LNMTL constitute a ray of hope
and indicate the practicability of literary MT. This section explores the
24 This refers to the process of fans translating and subtitling multimedia materials.
commercialised translation industry of Chinese internet literature and

reviews the features of Xianxia literature to further demonstrate how
MT works in this genre.
In 2004, the first paid-reading service, provided by the platform Qidian,
upset the entire Chinese internet literature market and marked its entry
into the new era of commercialisation. Among the accompanying changes,
three form the essential preconditions for MT used in internet literature:
the longer length of web novels, the lower entry barrier for writers and
the readers’ preference for quick updates. Firstly, Chinese net literature,
which used to take the form of novellas (300,000 to 1 million Chinese
characters), has morphed to form serials due to the emergence of the paid-
reading service where authors are paid by word count, entailing the promo-
tion of a longer length.25 Novels (about 2 to 4 million Chinese characters)
have thus replaced novellas as the mainstream form. Secondly, there are
fewer average requirements for authors writing web novels than for authors
writing formal literature; in cyberspace everybody has a chance to become
a writer. Quantity, instead of quality, is what many net literature writers
and profit-oriented websites tend to value most, with thousands of titles
being published online and hundreds of new updates every day. The market
is profit-oriented and caters to readers and is expanding rapidly. Thirdly,
the lightning speed of updates of the original novels would indicate that
prompt translations constitute the linchpin by which foreign readership
is captured and maintained. According to iResearch (2017, p. 36), when
deciding which novel to read next, 82% of readers valued the update rate
most, while 76.1% valued translation quality. The volume of daily updates
varies from author to author. Generally speaking, the usual speed is about
6,000 Chinese characters per day, but for some top-rated writers or popular
novels, it can reach 10,000 (Ji, 2016). It is impossible for any literary trans-
lator to keep to the same pace as the original writer, especially those who
translate from their mother tongue into a foreign language, not to mention
that the cost of human translators is much higher than for machines.
As noted above, when translating Xianxia literature, efficiency is priori-
tised over quality. Despite putting quality at risk in some cases, MT can
provide an acceptable translation service when there is a lack of money,
time and human resources. Even if machines are baffled by culture-specific
terms and metaphorical expressions (see Chapter 8 for a systemic analysis
of MT and HT of metaphor in literary texts) or fail to interpret traditional
Chinese characters, humans can still, at the very least, intervene to improve
25 Chinese internet literature adopts a different length system than print literature. An online
novel usually contains more than 3 million Chinese characters (Pei, 2022). However, print
literature, for example, the Mao Dun Literature Prize, the most prestigious literature prize
in China would categorise a piece above 130,000 Chinese characters as a novel.
154 Shuyin Zhang
the translations by post-editing them. For example, hyperlinks are one way
of sharing additional information to inform readers of translation chal-
lenges and explain any cultural implications inherent in the material. These
hiccups seem not to bother the readers of Xianxia literature much accord-
ing to their feedback on Novel Updates. Some giant Chinese internet com-
panies have noticed the merits of MT and have consequently started to
invest in researching and developing AI translation for net literature. A
prime example is funstory.ai,26 a Chinese company that offers AI trans-
lation services for internet literature platforms and currently has a very
significant daily update volume, with over 2,000 works updated simulta-
neously, covering more than 180 countries and regions (Chai, 2019).
In contrast, the intrinsic characteristics of Xianxia novels that make the
genre popular, discussed in Section 7.1—its unique terminology, straightfor-
ward language and relatively fixed plot pattern with its feel-good storyline—
also support the feasibility of MT. Notably, the genre uses a large amount
of terminology that consists of culture-specific or genre-specific Chinese
elements. Writers sometimes also devise terms to create a new world, but
the cultivation systems referred to in the novels largely stem from Daoism,
which means there will always be some regular or predictable terms, neces-
sitating the use of a terminology database within one novel and across dif-
ferent novels. Examples include 因果 (karma), 轮回 (samsara), 五行 (five
elements) and 阴阳 (yin and yang) to name a few. Meanwhile, the readership
is not expecting serious literature. ‘Research suggests that most foreign read-
ership of Chinese internet literature constitutes unemployed males (students)
under the age of 30, whose reading purpose is more for entertainment and
pleasure than for literary appreciation’ (iResearch, 2017, pp. 25–8). The
genre gives the readers an utterly unconventional experience and demands
no hard work on the part of a translator, so that there is room for machines,
even though the extraction of terminology and the maintenance of transla-
tion memories requires extra effort, and the formatting, linguistic and cul-
tural issues still need human intervention.
While the translation market for Xianxia literature is rapidly expand-
ing, many translation platforms are still struggling to recruit professional,
experienced literary translators. Unlike translating formal literature, trans-
lating internet literature does not give translators the prestige required to
offset their low income, which makes recruitment even harder. In the mean-
time, MT is becoming progressively more prevalent in different industries
and everyday life through technological advancements and internationali-
sation, and the ready availability of MT engines gives readers a chance to
26 https://funstory.ai/.
switch their roles from readers to translators and editors, thus establishing
a more intimate relationship with a text. This trend is also applicable to
certain literary genres, undesirable as it might seem to many professional
translators and literary critics, who, in the past, have castigated MT as
unsuitable for literary translation.
7.5 Conclusions
The gap between the fast-growing overseas market for Xianxia novels and
their ensuing translation activity and the time lag for MT applied in liter-
ary translation highlights the necessity for research in this area. The present
chapter explores the potential and practicalities of literary MT by reviewing
Chinese internet literature and the characteristics of the Xianxia genre and
analysing the translation models used by leading platforms. Xianxia literature,
which epitomises commercialised Chinese net literature, with its specific termi-
nology, reduced literary quality and set patterns, complies with the pervasive
use of interactive MT in the digital age and international dissemination of
Chinese culture. The MTPE model, which cuts costs, saves time and main-
tains quality to a large extent, can contribute to fan-based and commercial
websites. The future of literary MT is promising as it is establishing a more
intimate relationship between readers, translators and texts, and creating a
channel for better communication between different cultures while at the same
time enhancing cooperation between humans and machines.
However, MT still has a long way to go in terms of literary translation,
even with pre- and post-editors handling cultural and linguistic issues.
There are still insurmountable gaps for machines to overcome, at least for
now. Technical constraints are primary concerns in this regard, but some
ethical issues also need to be addressed to promote literary MT. The most
urgent questions are: will MT shake the status quo enjoyed by profes-
sional literary translators? Will machine-translated literature be regarded
as inferior to human-translated literature—a ‘secondary’ work’s ‘second-
ary’ work? If so, is it offensive to the original author if someone machine-
translates their work? Does the translation industry still need human
translators for particular net literature genres or will human editors suf-
fice? Apart from answering these questions, empirical research involving a
comparison of different MT outputs of an internet novel also needs to be
conducted in support of literary MT.
References
Chai, R. (2019). Ta yong AI fanyi wangluo xiaoshuo shouge 30 yi yangfensi
他用AI翻译网络小说收割30亿洋粉丝 [AI translated web novels attract 3 billion
foreign fans]. https://funstory.ai/2019/09/24/
156 Shuyin Zhang
Drugan, J. (2013). Quality in professional translation: Assessment and

improvement. Bloomsbury. https://www.bloomsbury.com/uk/quality-in
-professional-translation-9781441149541/
Duan, G. (2018). Chinese internet literature. In M. Gu & T. Feng (Eds.), Routledge
handbook of modern Chinese literature (pp. 669–81). Routledge. https://doi
.org/10.4324/9781315626994
Hayles, N. K. ( 2008). Electronic literature: New horizons for the literary.
University of Notre Dame Press. https://undpress.nd.edu/9780268030858/
electronic-literature/
Hutchins, W. J. (1995). Machine translation: A brief history. In E. F. K. Koerner &
R. E. Asher (Eds.), Concise history of the language sciences: From the Sumerians
to the cognitivists (pp. 431–45). Pergamon Press. https://doi.org/10.1016/B978
-0-08-042580-1.50066-0
iResearch. (2017). 2017 nian Zhongguo wangluo wenxue chuhai baipishu
2017年中国网络文学出海白皮书 [2017 Chinese online literature going global]
[White Paper]. http://report.iresearch.cn/report/201709/3057.shtml
iResearch. (2020). 2020 nian Zhongguo wangluo wenxue chuhai yanjiu baogao
2020年中国网络文学出海研究报告 [2020 Report on Chinese online literature
going global]. https://report.iresearch.cn/report/202008/3644.shtml
Jensen, K. B. (2016). Intermediality. In K. B. Jensen, R. T. Craig, J. D. Pooley, &
E. W. Rothenbuhler (Eds.), The international encyclopedia of communication
theory and philosophy (pp. 1–12). https://doi.org/10.1002/9781118766804
.wbiect170
Ji, X. (2016, December 13). Wangluo xieshou xianzhuang 网络写手现状 [The
status quo of Internet writers]. Chinanews. http://www.chinanews.com/cul
/2016/12-13/8091809.shtml
Jia, M. (2020, November 20). Growing overseas readership of Chinese online
literature. China Daily. https://www.chinadaily.com.cn/a/202011/20/WS5fb79
4e8a31024ad0ba957fa.html
Jiang, Y., & Yu, B. (2017). A contrastive study on the rendition of adjectival
possessive pronouns in Pride and Prejudice by human translation and online
machine translation. Journal of Xidian University (Social Science Edition),
27(2), 147–55. https://doi.org/10.16348/j.cnki.cn61-1336/c.2017.02.021
Jones, R., & Irvine, A. (2013). The (un)faithful machine translator. Proceedings
of the 7thWorkshop on Language Technology for Cultural Heritage, Social
Sciences, and Humanities (LaTeCH 2013), Sofia, Bulgaria (pp. 96–101).
Association for Computational Linguistics. https://aclanthology.org/W13
-2713/
Kenny, D. (2019). Machine translation. In M. Baker & G. Saldanha (Eds.),
Routledge encyclopedia of translation studies (3rd ed., pp. 305–10). Routledge.
https://doi.org/10.4324/9781315678627
translator’s voice. Translation Spaces, 9(1), 123–49. https://doi.org/10.1075/ts
.00024.ken
Li, Y. (2019). The translation of Chinese webnovels in the virtual community: A
netnographics study of Wuxiaworld [Unpublished PhD dissertation]. Shanghai
International Studies University, CNKI. https://kns.cnki.net/KCMS/detail/detail
.aspx?dbname=CDFDLAST2020&filename=1019111045.nh
O’Hagan, M. (2009). Evolution of user-generated translation: Fansubs, translation

hacking and crowdsourcing. The Journal of Internationalization and
Localization, 1(1), 94–121. https://doi.org/10.1075/jial.1.04hag
Ouyang, Y. (2008). Wangluo wenxue gailun 网络文学概论 [Introducing internet
literature]. Beijing University Press.
Pei, Z. (2022). 为什么在快节奏的时代网络小说这么长 [Why are web novels so
long in the fast-paced era]. 文艺报 [Wen Yi Bao].
Rettberg, S. (2019). Electronic literature. Polity Press. https://www.wiley.com/en
-gb/Electronic+Literature-p-9781509516773
Ruan, S. (2019). China’s web novels: Overseas spread and implications to
translation studies. Journal of Yanshan University, 20(1), 11–18.
Shao, Y., Ji, Y., & Xiao, Y. (2018). Meijie geming shiye xia de Zhongguo wangluo
wenxue haiwai chuanbo 媒介革命视野下的中国网络文学海外传播 [Overseas
dissemination of Chinese internet literature from the perspective of media
revolution]. Theory and Criticism of Literature and Art, 2, 119–29. https://doi
.org/10.16532/j.cnki.1002-9583.2018.02.011
Shih, C. (2016). Can machine translation declare a new realm of service? Online
folktales as a case study. Theory and Practice in Language Studies, 6(2), 252–9.
http://www.academypublication.com/issues2/tpls/vol06/02/05.pdf
Taivalkoski-Shilov, K. (2018). Ethical issues regarding machine(-assisted)
translation of literary texts. Perspectives, 27(5), 689–703. https://doi.org/10
.1080/0907676X.2018.1520907
Toral, A., & Way, A. (2014). Is machine translation ready for literature? In J.
Esteves-Ferreira, J. Macan, R. Mitkov, & O.-M. Stefanov (Eds.), Proceedings
of Translating and the Computer 36 (pp. 174–6). https://aclanthology.org/2014
.tc-1.23.pdf
Voigt, R., & Jurafsky, D. (2012). Towards a literary machine translation: The role of
referential cohesion. In D. Elson, A. Kazantseva, R. Mihalcea, & S. Szpakowicz
(Eds.), Proceedings of the NAACL-HLT 2012 Workshop on Computational
Linguistics for Literature, Montréal, Canada (pp. 18–25). Association for
Computational Linguistics. https://aclanthology.org/W12-25.pdf
Wang, Z., & Xie, T. (2013). Introducing Chinese literature and culture: Puzzles
and reflections. Academic Monthly, 45(2), 21–7. https://doi.org/10.19862/j
.cnki.xsyk.2013.02.003
Wu, J., Zhang, L., & Zhu, H. (2018). Jiqi fanyi yu rengong fanyi zhi chayi –
Yi wenxue wenben wei li 机器翻译与人工翻译之差异—以文学类文本为例
[Difference between machine translation and human translation: A case study
on literary texts]. Overseas English, 16, 148–58.
Yang, L. (2019). A study on China’s image constructed by English translations
on online fantasy novels—A case study of Xianxia genre [Unpublished PhD
dissertation]. Shanghai International Studies University, CNKI. https://kns
.cnki . net / KCMS / detail / detail . aspx ? dbname = CDFDLAST2020 & filename
=1019114702.nh
Zhu, L. (2020). A report on the C-E translation strategies of culture-loaded
words in the fantasy novel apotheosis (excerpts) from the perspective of
reception aesthetics [Unpublished Master’s dissertation]. Liaoning University,
CNKI. https://kns.cnki.net/KCMS/detail/detail.aspx?dbname=CMFD202101
&filename=1020824601.nh
8 Up and About, or Betwixt and
Between?
The Poetry of a Translation Machine
Tim Van de Cruys
8.1 Introduction
Computers have made remarkable strides in processing natural language in
recent years. Machine translation (MT) applications are advancing rapidly,
and the newest language models enable computers to generate coherent
and meaningful text autonomously. In this article, we will delve into the
developments underlying these advances, and explore how the algorithms
can be augmented for automatic poetry generation and translation.
About 70 years ago, the first algorithms for automatic natural language
processing (NLP) appeared as part of the emerging field of artificial
intelligence (AI). Over time, several different algorithms have emerged,
which can be broadly classified into three paradigms: symbolic, statistical,
and neural networks.
The symbolic paradigm, which is the oldest, uses handwritten rules to
model language. Generative grammar, developed by Chomsky (1957), is
an example of a rule-based system for NLP. Rule-based systems have also
been used in MT to transform sentences from the source language to the
target language. However, designing these rules can be a tedious process,
and their coverage is limited.
The statistical paradigm (Koehn, 2009), which emerged in the late
1980s, aims to equip computers with self-learning capabilities for NLP.
The goal of statistical models is to learn a model of language from large
text collections and perform tasks such as MT automatically. In contrast
to the symbolic paradigm, which relies on manually designed rules, the
statistical approach allows for a more automatic approach to NLP.
The most recent and successful models for automatic language processing
use neural networks (Goldberg, 2017). Although neural networks were
first proposed in the 1950s, their use in NLP has only become widespread
in the last decade. Neural networks are a type of machine learning, where
the model learns from training data to make predictions for new data.
However, unlike the statistical model, neural networks possess the ability
to build complex representations based on the input data, leading to
DOI: 10.4324/9781003357391-11
Up and About, or Betwixt and Between? 159
the best possible prediction. This flexibility allows us to speak of a new

paradigm for NLP.
Note that contemporary NLP systems are exclusively task-oriented:
a neural network is trained on a significant amount of training data,
optimizing the parameters of the model to generate the best possible
output for a given task or problem. In this way, the network merely mimics
human language, leaving little room for creativity. Might it still be possible
to elicit some form of creativity from the network? Paradoxically, we can
achieve this by imposing certain constraints on the network, so it must look
for new ways to perform its task. This is the approach we will examine for
poetry generation and translation.
This chapter is structured as follows. In the next section, we will pro-
vide an overview of the neural techniques used for modelling language
applications, including language generation and MT. In Section 8.3, we
will demonstrate how standard neural networks can be adapted for poetry
generation. In Section 8.4, we will do the same for automatic poetry trans-
lation. Section 8.5 concludes and provides a number of avenues for future
work.
8.2 Neural Networks for NLP

8.2.1 Introduction
Neural networks are an approach to machine learning that draws

inspiration from the functioning of neurons in the human brain. In an
artificial neural network, different neurons are organized into layers, with
each neuron connected to those in the next layer, allowing the transmission
of signals between them (see Figure 8.1). Based on the input signals, each
neuron calculates a value, indicating its level of activation, which is then
transmitted to the next layer of neurons. The first layer is the input layer
that represents the language data, while the output layer is the prediction
of the network. In between these layers are the hidden layers, which
develop latent representations based on the input data to ensure accurate
predictions. This architecture allows basic features such as words to be
assigned a comprehensive meaning structure, known as word embeddings
(Mikolov et al., 2013), enhancing the generalization capabilities of the
network (see Section 8.2.2). The output layer represents the prediction of
the network.
Neural networks are a versatile learning mechanism that can be applied
to a wide range of applications such as image recognition, weather
forecasting, and predicting stock prices. However, the specific network
structure must often be adapted to suit the input data. In this section, we
discuss a number of techniques for the processing of language data.
160 Tim Van de Cruys
Figure 8.1 General structure of a neural network.
8.2.2 Word Embeddings
Word embeddings are one of the most important developments for neural
NLP. A monolingual word embedding is a vector of values that captures
a word’s similarity with the other words in a language, but multilingual
language-agnostic word embeddings are often used too. In both monolingual
and multilingual word embeddings, words with similar meanings have
values close to each other, while words with different meanings have values
further apart. Word embeddings offer an improvement over the statistical
paradigm, which represents words as atomic, indivisible features without
any underlying meaning structure.
One of the most attractive features of word embeddings is that they
are automatically derived from the data. Each word is assigned a set of
parameters, and the neural network is tasked with learning the appropriate
values for these parameters. A language model, which predicts the next word
in a text based on the previous words, is commonly used to learn embeddings.
Figure 8.2 shows a schematic representation of a network architecture for
inducing monolingual word embeddings. Matrix V contains the embeddings
for the entire vocabulary. The embeddings of the three preceding words are
selected as input to the network. Based on these inputs, the network induces
a hidden representation, and that representation is used to predict the
next word. The parameters of the network (including the embeddings) are
initialized with random weights, and the network automatically induces the
values that yield the best possible prediction. If the network assigns similar
embeddings to words such as ‘drink’ and ‘sip’, ‘a’ and ‘the’, and ‘white’ and
‘red’, it will be easier to make the correct prediction.
Figure 8.2 Architecture of a neural language model with word embeddings.
By training a neural language model to predict the next word based

on (say) the previous three words, word representations are automatically
induced from the data that capture the relationship between word mean-
ings. These word embeddings can be transferred to other network architec-
tures, providing models with knowledge of word meaning, allowing them
to make accurate predictions with limited training data.
8.2.3 Recurrent Neural Networks
The network model described earlier performs well for the induction of
word embeddings. However, it can also be used as a language generation
model, where the goal is to predict the next word in a sentence based
on the preceding three words. The disadvantage of this approach is that
the network’s structure is fixed, requiring a fixed context of three words.
It would be better if the entire sentence context could be considered for
predicting the next word. This is often achieved using a recurrent neural
network (RNN), which allows for the entire sentence context to be captured
(Mikolov et al., 2010). A graphical representation of such a network can
be found in Figure 8.3.
In an RNN, a sentence is read word by word, and the hidden
representation of the network calculated in the previous step is reused in
the next step. To illustrate how the generation process works, we will walk
through several calculation steps. Suppose we start a sentence with the word
‘I’. In this case, we take the embedding for the word ‘I’ (represented as x)
and input it into the network, along with an initial context representation
a0 indicating that we are at the beginning of a sentence. Based on these
two representations, a hidden representation (a1) is calculated, indicating
that we have seen the word ‘I’ at the beginning of the sentence. This
hidden representation is then used to predict a probability distribution
that reflects the likelihood of each word in our vocabulary. We select a
word with a high probability (such as ‘see’) as the next word. We then
repeat the same calculation with the previously selected word as input. We
take the embedding for the word ‘see’ and input it into the network, along
with the context representation we built in the previous step (meaning
that representation a1 replaces a0). Based on this input, we calculate a
new hidden representation that now includes the sentence context ‘I see’.
This hidden representation leads to a new probability distribution, based
on which we can choose the next word. In this way, we build a sentence
word by word, where the hidden representation of the network (through
the recurrent connection) always represents the full-sentence context,
allowing us to consider the entire context seen so far for predicting the
next word.
Figure 8.3 A recurrent neural network for language generation.

An RNN can be used not only to generate language but also to build
a representation for an existing sentence. To do this, we read the existing
sentence word by word into the network, and the recurrent connection
gradually builds the representation of the sentence. We can then use this
final representation for a prediction regarding a specific language-pro-
cessing application (such as sentiment analysis), or to encode a source
sentence into a representation that is used to predict a target translation
(Section 8.2.5).
8.2.4 Transformers
One of the most influential developments in the field of neural NLP is

the Transformer (Vaswani et al., 2017), which is capable of building a
precise representation of a complete sentence based on contextualized
word embeddings. Intuitively, a Transformer architecture ensures that the
representation of a word is adapted to its sentence context: each word
looks at all the other words in the sentence and determines how it should
be adapted to represent its meaning in the context of the other words.
This adjustment occurs by using a self-attention mechanism throughout
the different layers of the network. Transformers were initially introduced
in the context of MT, but they can equally be used for text generation.
Based on the contextualized representations, the next word in the sentence
is predicted, and the newly generated word is integrated into the network
for the prediction of the next word. Modern Transformer models,
commonly known as large language models (LLMs), often contain billions
of parameters and are trained on massive amounts of data. A well-known
Transformer model for language generation is GPT-3 (Brown et al., 2020).
It is equally the underlying architecture for the ChatGPT model that is the
current flavour of the month.
8.2.5 Neural Machine Translation
Until around 2015, the statistical method provided the best MT quality.
However, the emergence of the neural paradigm quickly changed this (see
Section I.2, Introduction, this volume, for a more historical perspective).
A neural MT (NMT) system considers the translation process as a
large mathematical function, modelled using neural networks. The
NMT system consists of two networks: an encoder and a decoder (see
Section I.2, Introduction, this volume, for more explanation of how
NMT systems are built). The encoder builds a representation of the
sentence in the source language, which is then passed on to the decoder.
The decoder then decodes that representation word by word into the
target language. RNNs can be used as an underlying architecture for
the encoder and decoder, in which case two RNNs are linked together.
The first network, the encoder, sequentially processes the words of the
sentence in the source language, building up a hidden representation.
This representation is then passed as the initial hidden representation
to the second network, the decoder. Based on that representation, the
decoder predicts word by word the translation of the sentence in the
target language. The model is trained on a parallel text corpus. The
sentence in the source language serves as input to the encoder, and based
on the resulting representation, the decoder is tasked with reproducing
the sentence in the target language as correctly as possible by adjusting
the weights of the entire network. Instead of RNNs, Transformers can
also be embedded in an encoder-decoder architecture, but using a simpler
feed-forward network as in Figure 8.1. This is currently the method that
produces the best possible MT quality, although LLMs like GPT-3 are
likely to provide strong competition to these bespoke NMT systems in
the near future.
8.3 Automatic Poetry Generation

The automatic generation of poetry presents a significant challenge. The
creation of a meaningful poem requires the consideration of both linguistic
and literary aspects. The former involves a proper modelling of language,
which encompasses syntactic well-formedness and topical coherence.
The latter involves the incorporation of various constraints, such as form
and rhyme, which are specific to a particular poetic genre. Moreover, a
successful poetry-generation system must exhibit a degree of literary
creativity that renders the poem interesting and valuable.
One possible approach to automatic poetry generation using neural
networks is presented in Van de Cruys (2020), which makes use of RNNs in
an encoder-decoder configuration. The encoder generates a representation
of an entire sentence by sequentially incorporating each word of the
sentence into a fixed-size hidden state vector. This representation is then
passed to the decoder, which generates a sequence of words according to a
probability distribution derived from the input sentence’s hidden state. By
training the network to predict the next sentence with the current sentence
as input, the system learns to generate plain text with discourse coherence.
To introduce poetic constraints, we can modify the probability distribution
yielded by the decoder. This enables the incorporation of form and rhyme,
thereby conferring a poetic character to the generated verse. Notably,
the poetry-generation system is not trained on poetic texts. Instead, the
system is trained on a corpus of standard, prosaic texts extracted from the
web. Through the application of constraints to the network’s probability
distribution, the system can generate poetry that meets the requirements of
a specific poetic genre.
The system uses two types of constraints: a rhyme constraint and a

topical (subject-related) constraint. The rhyme constraint is based on a
phonetic representation of words, extracted from the online dictionary
Wiktionary.1 For each word, we determine its rhyme sound and the group
of consonants that precedes the group of vowels. The next step is to cre-
ate a probability distribution for a particular rhyme sound that the verse
should adhere to. This distribution is used as a prior probability distribu-
tion in order to reweight the neural network’s standard output probability
distribution.
The topical constraint is modelled using a latent semantic model based
on non-negative matrix factorization (NMF; Lee and Seung, 2000), an
approach which has been shown to be a useful decomposition for data
containing multiple variables. Previous research has shown that NMF
methods are able to induce clear-cut, interpretable topical dimensions
automatically from a large corpus of texts (Van de Cruys et al., 2011;
Murphy et al., 2012). This model can then be used to generate a poem
according to a particular topical dimension. Each word in the vocabu-
lary is given a weight according to a particular topical dimension. These
weights may then be used to reweight the neural network’s standard out-
put probability distribution, generating words that adhere to the topical
constraint. A graphical representation of the architecture for poetry gen-
eration is given in Figure 8.4.
Figure 8.4 An overview of the architecture for poetry generation.
1 https://en.wiktionary.org/wiki/Wiktionary:Main_Page.
The encoder encodes the current verse, and the final representation
is given to the decoder, which predicts the next verse word by word in
reverse. The rhyme prior is applied to the first-time step in the decoding
process, and the topic prior is optionally applied to all time steps. This
prevents the generation of an ill-chosen rhyme word that does not fit well
with the rest of the verse.
The model is used to generate a large number of candidate verses.
By using an optimization procedure, the best sample can be chosen
according to specific constraints. Each candidate is scored based on
various criteria, including language model score, compliance with rhyme
and topical constraints, and an optimal number of syllables. These scores
are normalized, and the harmonic mean of all scores is taken as the final
composite score for each candidate. The candidate with the highest score is
then chosen and added to the poem. This process ensures that the generated
verse meets the specific constraints and results in a high-quality poem.
An example of a poem generated by the system is given below.
As sad as it seems, it’s hardly ever made

Alas, there’s nothing much like evil spirits
Indeed, the words uttered will never fade
No wonder we have to face our own wits
But you can’t see the joy of life yet
Ah, my dear, my dear brothers and sisters
Why can’t you say to me the deepest regret
Why don’t you love me, my beloved strangers
When subjected to human evaluation, the system yields favourable results,

with about half of the generated poems judged as written by humans. On
a range of evaluation criteria, the system scores significantly better than
competing systems (Ghazvininejad et al., 2016; Lau et al., 2018). For more
details on the poetry generation system and its evaluation, the interested
reader is referred to Van de Cruys (2020).
8.4 Automatic Poetry Translation

Arguably, the translation of poetry from one language to another is an even
more complex task, as it involves not only preserving the meaning of the
original text but also maintaining its poetic essence. Rhyme is one of the
key elements of poetry, and it poses a significant challenge to automated
translation models. Traditional MT methods fail to capture characteristics
like rhyme, resulting in a literal translation of the poem.
Similar to the poetry generation system, our aim is to apply constraints
to a standard NMT setup in order to automatically translate poems. In
this section, we present some exploratory results on automatic poetry
translation using a Transformer-based encoder-decoder model adapted

from MarianMT (Junczys-Dowmunt et al., 2018; Lison et al., 2018; see
also Section I.2, Introduction, this volume).2 The model is trained on a
large parallel dataset of English and French sentences (150 million in
total). Similar to the constraint used in for poetry generation, we incorpo-
rate an a priori rhyme distribution to enforce rhyme in the target transla-
tion. However, imposing rhyme on the final position of each line can lead
to problems with syntax and fluidity, as the translation model does not
generate the translated rhymes in reverse. We therefore perform a search
for the optimal translation candidate. We generate a large number of can-
didates, imposing the rhyme on different sentence positions. We also make
use of a large beam of ten and keep all hypotheses, i.e. in searching through
the hypothesis space to try to come up with the ‘best’ solution, we keep
all candidates within a width of ten from the best one. We then order the
different candidate translations according to the score of the translation
model and pick the one with the highest probability. The result of this
approach yields the best translation the model could come up with given
the imposed rhyme constraint.
In order to qualitatively judge the performance of the resulting model,
we automatically translated the classical poem ‘There Will Come Soft
Rains’ by Sara Teasdale, an American poet born in 1884 and who died
in 1933. She was known for her lyrical and emotional poetry that focused
on themes of love, beauty, and nature. Teasdale won the Pulitzer Prize for
poetry in 1918 for her collection Love Songs. Her work has been widely
anthologized and remains popular today.
The original poem goes as follows:
There will come soft rains and the smell of the ground,
And swallows calling with their shimmering sound;
And frogs in the pools singing at night,
And wild plum-trees in tremulous white;
Robins will wear their feathery fire
Whistling their whims on a low fence-wire;
And not one will know of the war, not one
Will care at last when it is done.
Not one would mind, neither bird nor tree
If mankind perished utterly;
And Spring herself, when she woke at dawn,
Would scarcely know that we were gone.
2 https://marian-nmt.github.io/.
We can see the poem has a straightforward AABB-rhyming scheme, which

we would like to preserve in our translation into French. For comparison,
we will first automatically translate the poem with the default translation
system, without any constraints added. The translation by the default
system goes as follows:
Il arrivera des pluies douces et l’odeur du sol

[There will arrive gentle rains and the smell of the soil]
Et les hirondelles crieront avec leurs cris chatoyants
[And the swallows will cry with their shimmering cries]
Et les grenouilles dans les étangs chantent la nuit
[And the frogs in the ponds sing at night]
Et les pruniers sauvages dans un blanc tremblement
[And the wild plum trees in a white trembling]
Les merles porteront leur feu de plumes
[The blackbirds will carry their fire of feathers]
Sifflant leurs caprices sur un fil de clôture bas
[Whistling their whims on a low fence wire]
Et personne ne connaîtra la guerre, et personne
[And no one will know war, and no one]
Ne se souciera enfin quand elle sera faite
[Will finally care when it is done]
Nul ne s’en souciera, ni oiseau ni arbre
[No one will care, neither bird nor tree]
Si l’humanité périssait complètement
[If mankind perished completely]
Et le printemps lui-même, lorsqu’elle se réveilla à l’aube
[And spring itself, when she awoke at dawn]
Ne saurait à peine que nous étions partis
[Would hardly know that we were gone]
The translation is a fairly accurate, but very literal translation of the

original poem. The rhyme scheme of the original English poem is evidently
completely lost. Let us now translate the English poem again, but this time
we enforce the rhyme constraint, making sure that the resulting poem
adheres to the original AABB rhyme scheme:
Il y aura des pluies douces et l’odeur de la terre

[There will be gentle rains and the smell of the earth]
Et les hirondelles crient avec leur concert
[And the swallows cry with their concert]
Et les grenouilles qui chantent la nuit
[And the frogs that sing at night]
Et de pruniers sauvages dans un blanc tremblant ainsi

[And wild plum trees in a trembling white thus]
Les oiseaux chanteurs porteront leur feu plumeux
[Songbirds will carry their feathery fire]
Sifflant sur une clôture basse en fil de fer bleu
[Whistling on a low blue wire fence]
Et personne ne connaîtra la guerre, pas un seul animal
[And no one will know war, not a single animal]
Se souciera enfin quand elle sera finale
[Will finally care when it is final]
Cela ne dérangerait personne, ni arbre ni oiseau
[It would not bother anyone, neither tree nor bird]
Si l’humanité a péri dans tous ses maux
[If humanity has perished in all its ills]
Et le printemps elle-même, lorsqu’elle s’est réveillée au soleil
[And spring herself, when she woke up in the sun]
Savait à peine que nous étions partis, qu’il y avait une veille
[Barely knew we were gone, that there was a day before]
By using the rhyme constraint on the output probability distribution, we

notice that the original rhyme scheme is indeed preserved. Interestingly, the
system opts for a more liberal translation in order to properly enforce the
correct rhyme. As an example, fil de fer bleu [‘blue fence wire’] has been
changed from the original poem (‘low fence-wire’) in order to rhyme with
plumeux in the previous line. Moreover, the rhyme constraint causes the
translation for ‘feathery fire’ to be feu plumeux, yielding a more creative
translation than feu de plumes in the unconstrained version.
Generally speaking, the translation of the poem in the constrained version
is more liberal, to ensure that the form of the poem (i.e. the rhyme scheme)
is properly preserved. When translating a poem, an actual translator might
equally take some liberties to maintain the formal characteristics of the orig-
inal, which is not entirely dissimilar to the behaviour of the constrained MT
system described here. The prior probability distribution allows one to con-
trol the balance between semantic faithfulness (vis-à-vis the source poem)
and adherence to the poetic constraints. The prior probability distribution
can be designed to increase the probability of fitting rhyme words and lower
the probability of non-fitting ones. This allows the system to prioritize fitting
rhyme words while still selecting non-rhyme words that fit the semantics
of the source poem. Alternatively, the prior probability distribution can be
designed to solely allow rhyme words, resulting in a more liberal translation,
even if they do not entirely fit the semantics of the source poem.
Finally, one might ask oneself whether LLMs such as the recent ChatGPT
model have the inherent ability to accurately translate poetry, preserving
both the semantics and form of the source poem (see also Chapter 13.8,
this volume). Some preliminary experiments seem to indicate that this is not
the case. When prompted with the task of providing a literary translation
of the poem by Teasdale that preserves the rhyme of the original poem, the
ChatGPT model yields the following translation (prompted by the author
on 25th February 2023):
Il viendra des pluies douces et l’odeur de la terre,

[There will come gentle rains and the smell of the earth,]
Des hirondelles appelant de leur son chatoyant;
[Swallows calling with their shimmering sound;]
Des grenouilles dans les mares qui chantent la nuit,
[Frogs in the pools that sing at night,]
Des pruniers sauvages d’un blanc tremblan ;
[Wild plum trees of trembling white;]
Les rouges-gorges porteront leur plumage de feu
[The robins will wear their plumage of fire]
Sifflant leurs caprices sur un fil de clôture;
[Whistling their whims on a fence wire;]
Et nul ne saura rien de la guerre, pas un
[And no one will know anything about the war, not one]
N’en aura cure quand tout sera fini.
[Will worry when it is all over.]
Aucun ne s’inquiètera, ni les oiseaux, ni les arbres,
[No-one will worry, neither the birds nor the trees,]
Si l’humanité périssait complètement ;
[If mankind perished completely;]
Et le printemps lui-même, quand il se réveillera à l’aube,
[And spring itself, when it wakes at dawn,]
À peine saura-t-il que nous avons disparu.
[Hardly will know that we have disappeared.]
8.5 Conclusion and Future Work

NLP has rapidly advanced in recent years, with neural architectures at the
forefront of this progress. These models have greatly improved various
language tasks, including MT and language generation. Despite their
impressive performance, standard neural models merely reproduce the
data they are trained on and lack the ability to generate novel and creative
outputs.
To address this limitation, it is possible to make use of additional
constraints to guide the output of these models. By introducing such
constraints, it is possible to trigger a certain creative inclination in these
models, enabling them to produce more diverse and imaginative outputs
that fit the imposed poetic form. These constrained neural architectures
show promising results, successfully producing poetry, as well as
successfully translating existing poetry.
Despite these advances, fully automatic literary translation has not yet
arrived. While NMT systems can produce translations that are generally
accurate, they still struggle with more complex aspects of literary language,
such as idiomatic expressions and cultural nuances. However, constrained
neural architectures offer an attractive source of inspiration that can
provide authors and translators with ideas, giving rise to a multi-faceted
co-creative setup.
Moving forward, further exploration of these constrained models
may lead to even more significant breakthroughs for creative language
generation and translation. By incorporating additional constraints, such
as those derived from linguistic theory or cultural context, we can improve
the quality and creativity of language models and enhance their ability
to produce more complex and creative language. Ultimately, this could
lead to more accurate and nuanced translations, as well as new forms of
creative language generation that push the boundaries of what is possible
with current architectures.
References
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan,
A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger,
G., Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C.,
Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner,
C., McCandlish, S., Radford, A., Sutskever, I., Amodei, D. (2020). Language
models are few-shot learners. In Proceedings of 34th Conference on Neural
Information Processing Systems (NeurIPS 2020), Vancouver, Canada (pp.
1877–1901). https://dl.acm.org/doi/abs/10.5555/3495724.3495883
Chomsky, N. (1957). Syntactic structures. Mouton.
Ghazvininejad, M., Shi, X., Choi, Y., & Knight, K. (2016). Generating topical
poetry. In Proceedings of the 2016 Conference on Empirical Methods in
Natural Language Processing, Austin, Texas (pp. 1183–91). Association for
Computational Linguistics. https://aclanthology.org/D16-1126/
Goldberg, Y. (2017). Neural network methods for natural language processing.
Springer. https://link.springer.com/book/10.1007/978-3-031-02165-7
P18-4020/
Koehn, P. (2009). Statistical machine translation. Cambridge University Press.
https://doi.org/10.1017/CBO9780511815829
Lau, J.H., Cohn, T., Baldwin, T., Brooke, J., & Hammond, A. (2018). Deep-speare:
A joint neural model of poetic language, meter and rhyme. In Proceedings of
the 56th Annual Meeting of the Association for Computational Linguistics
(Volume 1: Long Papers), Melbourne, Australia (pp. 1948–58). Association for
Computational Linguistics. https://aclanthology.org/P18-1181/
Lee, D.D., & Seung, H.S. (2000). Algorithms for non-negative matrix factorization.
In Proceedings of 13th Conference on Neural Information Processing Systems
(NeurIPS 2000), Denver, CO (pp. 556–62). https://proceedings.neurips.cc/
paper_files/paper/2000/file/f9d1152547c0bde01830b7e8bd60024c-Paper.pdf
Lison, P., Tiedemann, J., & Kouylekov, M. (2018). OpenSubtitles2018: Statistical
rescoring of sentence alignments in large, noisy parallel corpora. In Proceedings
of the 11th International Conference on Language Resources and Evaluation
(LREC 2018), Miyazaki, Japan (pp. 1742–8). European Language Resources
Association (ELRA). https://aclanthology.org/L18-1275/
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010).
Recurrent neural network based language model. In Proceedings of Interspeech
2010, 11th Annual Conference of the International Speech Communication
Association, Makuhari, Chiba, Japan (pp. 2045–8). https://www.researchgate
.net / publication / 221489926 _ Recurrent _ neural _ network _ based _ language
_model
Mikolov, T., Yih, W.-T., & Zweig, G. (2013). Linguistic regularities in continuous
space word representations. In Proceedings of the 2013 Conference of the
North American Chapter of the Association for Computational Linguistics:
Human Language Technologies, Atlanta, Georgia (pp. 746–51). Association
for Computational Linguistics. https://aclanthology.org/N13-1090/
Murphy, B., Talukdar, P., & Mitchell, T. (2012). Learning effective and interpretable
semantic models using non-negative sparse embedding. In Proceedings of
COLING 2012, Mumbai, India (pp. 1933–50). https://aclanthology.org/C12
-1118.pdf
Van de Cruys, T., Poibeau, T., & Korhonen, A. (2011). Latent vector weighting for
word meaning in context. In Proceedings of the 2011 Conference on Empirical
Methods in Natural Language Processing, Edinburgh, Scotland (pp. 1012–22).
Association for Computational Linguistics. https://aclanthology.org/D11-1094/
Van de Cruys, T. (2020). Automatic poetry generation from prosaic text. In
Proceedings of the 58th Annual Meeting of the Association for Computational
Linguistics, Online (pp. 2471–80). Association for Computational Linguistics.
https://aclanthology.org/2020.acl-main.223/
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L.L., Gomez, A.N.,
Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In Advances
in Neural Information Processing Systems 30: Annual Conference on Neural
Information Processing Systems 2017, Long Beach, CA (pp. 5998–6008).
9 Metaphor in Literary Machine
Translation
Style, Creativity and Literariness
Aletta G. Dorst
9.1 Introduction
Metaphor and translation are no strangers to each other. In fact, they share
not only an etymology (metaphor originating from Greek meta [change,
e.g. of place] and pherein [to carry] and translate from Latin trans [across]
and ferre [to carry; past participle latus]) but also their main concepts and
terminology (e.g. source, target, substitution and transfer). And although
there is a relatively steady stream of studies on metaphor in translation to
be found in the academic literature (see Schäffner, 2017 and Shuttleworth,
2017 for useful overviews), these studies have tended to focus either on
offering theoretical deliberations on the translatability of metaphor (e.g.
Nida, 1964; Dagut, 1976) or on providing models for metaphor translation
in terms of translation procedures proposed for different types of metaphor
(e.g. Van den Broeck, 1981; Newmark, 1988).
Despite the fact that as academic disciplines both Metaphor Studies
and Translation Studies have undergone the same developmental phases
from an initial focus on language in ‘traditional approaches’, followed by a
cognitive and cultural turn in the 1980s, many researchers from Translation
Studies still approach metaphor as a predominantly linguistic phenomenon
(cf. Burmakova & Marugina, 2014; Dagnev & Chervenkova, 2020),
focusing heavily on unique and creative instances of metaphor. Likewise,
many researchers from Metaphor Studies still treat translation as a rather
simplistic process of linguistic substitution, disregarding extratextual
factors that influence the translation process and the decisions translators
make.
Metaphor translation can be approached from a linguistic or conceptual
perspective, as illustrated by the models developed by Newmark (1988) and
Schäffner (2004). Many studies focus on the type of linguistic metaphor
involved and whether the languages involved provide equivalent metaphors
(e.g. Newmark, 1988). More cognitively oriented studies consider whether
the cultures involved employ similar or different conceptual metaphors
to understand a particular phenomenon (e.g. Schäffner, 2004) or on
DOI: 10.4324/9781003357391-12
174 Aletta G. Dorst
whether similar or different mapping conditions apply (e.g. Mandelblit,

1995). However, none of the existing theories or models, nor the empirical
research based on them, seem to pay much attention to the perspective of
metaphor as a rhetorical device and an important tool in creating stylistic
coherence throughout a text. Nor do the overviews of how particular
procedures or approaches relate to different linguistic categories or
conceptual structures investigate how textual patterns of metaphor may
be affected by the translator’s local and global decisions. The stylistic
properties, purposes and effects of metaphor have, it seems, been largely
overlooked or taken for granted. Yet translation decisions based solely on
a metaphor’s linguistic and conceptual properties without consideration of
its communicative purpose may lead to disruptions in stylistic coherence
(e.g. Dorst, 2018).
Dorst (2018) argues that disruptions to the stylistic coherence of
metaphor patterns can be related to Berman’s (2021 [1985]) notion of
deforming tendencies, Venuti’s (2021) concept of domestication and
Toury’s (1995) law of growing standardisation. Whether wittingly or
not, translators may well be altering patterns of metaphor in literary texts
to conform to the norms of the target language and culture, diminishing
and deleting the metaphors’ stylistic and creative potential for the sake
of fluency and idiomaticity. The question asked in the current chapter
is whether a comparison between human and machine translation of
metaphor in literary texts can provide us with more insight as to whether
a translation process that is driven by statistical computation rather than
norm-governed behaviour leads to metaphor translation that is either
more stylistically accurate or detrimental to the text’s aesthetic value.
While human translators are more likely to be creative in their solutions,
the question remains as to whether this has a positive or negative effect
on the creativity of the metaphors (when original metaphors are replaced
with conventional ones), as well as the creativity of the text, i.e. when
creative solutions make the text idiomatic rather than stylistically
foregrounded.
9.2 Metaphor in Literary Translation

Metaphors have traditionally been considered a notorious problem
in translation as they are both linguistically and culturally embedded.
Newmark (1988, p. 104) even went as far as to consider metaphors ‘the
most important particular problem in translation’. Many scholars in
Translation Studies have considered the translatability of metaphor and
the different approaches that can be taken based on the type of metaphor in
question. In general, most settle on three approaches: 1) direct translation:
turning a metaphor into the same metaphor; 2) substitution: turning a
Metaphor in Literary Machine Translation 175
metaphor into a different metaphor; and 3) paraphrase: a shift to a non-

figurative equivalent.
Newmark (1981, pp. 88–91) distinguishes no fewer than seven different
procedures, which he lists in order of preference:
• Reproducing the same image in the TL

• Replacing the image in the SL with a standard TL image
• Translating metaphor by simile
• Translating metaphor by simile plus sense
• Converting metaphor to sense
• Deleting the metaphor
• Translating the same metaphor combined with sense
Newmark’s (1988, pp. 106–113) model, and most of the empirical

work based on it, uses a linguistic classification of metaphor types—into
dead, cliché, stock, adapted, recent and original—to determine which
translation procedure is preferred. To Newmark, only dead metaphors
are truly translatable. Both Van den Broeck (1981) and Alvarez (1993)
see conventional metaphors as the ‘most translatable’ ones, while they
consider original metaphors to be extremely difficult to translate, and stock
metaphors to be translatable only if the two cultures are closely related
(Alvarez, 1993, p. 137). Similarly, Al-Hasnawi (2007, n.p.) maintains
what determines how difficult a metaphor is to translate is not the presence
or lack of lexical equivalents in the languages involved, but ‘the diversity
of cultural conceptualization of even identical objects or worlds in both
communities whose languages are involved in translation’.
In recent studies focusing specifically on literary translation (e.g. Ali,
2006; Park, 2009; Chita & Stavrou, 2020; Dagnev & Chervenkova, 2020),
Newmark’s list of procedures is still the most commonly used method of
analysis. Chita and Stavrou (2020, p. 128) use it to analyse the translation
of Wilde’s Picture of Dorian Gray into German and Greek, showing that
the translators usually reproduced the image of the metaphors in their
translations, opting for retention over substitution or deletion perhaps as
a strategy to ‘preserve the accuracy of the literary style’. However, they
note that ‘in many cases the translators’ intent to reproduce the metaphors
in the target language was not successful’ (ibid.) and argue that the
translators were possibly hesitant to translate more freely out of respect
for the canonical source text.
A similar point is made by Park (2009) on the basis of his analysis of five
Korean translations of the short story Ligeia by Poe. Park demonstrates that
the transfer of meaning takes priority over the transfer of form, arguing
that ‘[w]hen the semantic translation cannot help T[arget]T[ext] readers
understand the metaphor, the translator should try to find equivalence
176 Aletta G. Dorst
rather than identity’ (p. 155). In such cases, translators should opt for any
of the seven procedures proposed by Newmark except the first (retention of
the SL image). Park emphasises that in order to ‘make a proper metaphor
translation, translators should recognize that metaphor translation needs
relevant knowledge and substantial background study’ (p. 170).
Dagnev and Chervenkova (2020) identified the conceptual metaphors
underlying metaphorical linguistic expressions in five canonical works—
Joseph Conrad’s Heart of Darkness, Virginia Woolf’s Mrs. Dalloway,
George Orwell’s 1984, Richard Russo’s Empire Falls and James Joyce’s
The Dead—and how they were translated into Bulgarian to determine
whether metaphors are lost or transformed. Distinguishing between
‘sleeping metaphors (lexicalised—non-transparent)’, ‘linguistically
expressed conceptual metaphors (entrenched—conventional and activated
in the text)’ and ‘creative metaphors (novel)’ (p. 103), and focusing on
whether they occur in the same or a different mapping condition, they
found that lexicalised metaphors were more often transformed than
sleeping or creative ones, most likely due to their culture-specific nature
and their linguistic entrenchment. Creative metaphors translated most
readily, while foregrounded metaphors, grammatical metaphors and
culturally bound ones turned out to be the ones most difficult to translate
(p. 110). Contrary to their expectations, they did not find many instances
of metaphor explication, paraphrase or loss, and the target texts were not
inferior to the source texts in terms of their metaphor depth and variety,
suggesting that ‘Toury’s law of growing standardization (1995) is not
applicable to the translation of highbrow literature by well-established and
experienced literary translators’ (ibid., p. 110).
This last claim is interesting in light of the common distinction between
experienced and novice translators. Ali (2006) looked specifically at
students training to become literary translators. Starting from Newmark’s
list of procedures, he warns against ‘the dangers involved in the application
of the set of translation procedures proposed by some translation theorists
as “alternative solutions” in cases where a given SL metaphor does not lend
itself to being retained in a TL’ (p. 134). Ali shows that Newmark’s model
does not make clear or explicit how translators should choose between the
different procedures, ‘thus turning metaphor translation into a random
process’ (ibid.). He argues that translators need a solid understanding
of the complex nature of metaphor and the many different forms and
functions it may have in authentic discourse. They also need to understand
the relevance of the metaphor in both the source and target culture, as
well as their own personal views on the purpose of their translation, i.e.
‘whether the aim is to produce a semantic translation (loyal to the SL
culture) or a communicative translation (primarily oriented towards the
TL reader)’ (ibid).
This brings us to the focus of the current chapter, as it suggests that for
the translation of metaphor in literature to be ‘correct’ or ‘appropriate’,
an MT system would need to adhere to an intricate set of rules and
guidelines that capture both the complex nature of metaphor as well as
the complex nature of translation. A small but growing number of studies
(e.g. Genzel et al., 2010; Green et al., 2010; Voigt & Jurafsky, 2012; Jones
& Irvine, 2013; Besacier, 2014; Toral & Way, 2015a, 2015b, 2018) has
investigated the usefulness of statistical and neural machine translation
(NMT) systems for literary translation (see Introduction of this volume
for a more detailed discussion) and obtained promising results. However,
no studies have thus far focused specifically on metaphor, so we do not at
present know whether the perceived quality of literary MT or the errors
observed are in any way related to the presence of particular types or uses
of metaphor. Nevertheless, a recent study by Guerberof-Arenas and Toral
(2022) suggests that metaphor is one important unit of ‘creative potential’
in literary texts, and as such, it is expected that metaphors may pose a
problem in translation that requires a creative solution, while MT tends
to be direct, which in this case means that it will retain the metaphor by
default.
9.3 Comparing Human and Machine Translations of

Metaphor in Literature
The comparison below is based on a short excerpt taken from Margaret
Atwood’s The Handmaid’s Tale (1985). Eight students taking an advanced
MA course in Literary Translation at Leiden University translated the
opening paragraph (from ‘A chair, a table, a lamp.’ to ‘Why do I want?’
(1996, p. 17), 191 words) and closing paragraph (from ‘Better her than
me…’ (1996, p. 20) to ‘Why tempt her to friendship?’ (1996, p. 21),
603 words) of Chapter 2 ‘Shopping’ (794 words in total) into Dutch for
their final assessment. No specific instruction or reading had been provided
during the course on Metaphor Studies or metaphor translation. I was the
course instructor and selected these fragments because they illustrate the
novel’s complex style and contain both highly conventionalised and thus
less noticeable uses of metaphor alongside very striking and original uses. I
was curious to see whether the students would approach the conventional
metaphors differently from the original ones. In the present chapter, the
eight student translations are compared to the official Dutch translation
(De Blaauw, 1987) and output produced by Google Translate’s NMT
system.
As already mentioned, the excerpt contains a number of striking and
original metaphors. In line with the findings of Dorst (2011, 2015), these
creative metaphors are all expressed as similes. The fact that this short passage
has no fewer than four similes in close proximity (a phenomenon called
178 Aletta G. Dorst
‘clustering’), and the fact that they are all extended and creative is particularly
noteworthy seeing as the number of similes is generally low even in fiction
(Steen et al., 2010; Dorst, 2011). The four similes in the relevant passage
(1996, p. 20) are also semantically related via ‘bodies’—‘heads’—‘voices’:
(1) […] all the different kinds of mischief that our bodies, like unruly
children, can get into.
(2) We would nod our heads as punctuation to each other’s voices, sign-
aling that yes, we know all about it.
(3) [;] gently we would complain, our voices soft and minor key and
mournful as pigeons in the eaves troughs.
(4) I hear where you're coming from, as if the voice itself were a traveler,
arriving from a distant place.
Not surprisingly, Google Translate retains all of these metaphors and pro-
duces a direct translation that follows the sentence structure and word
order very closely:
[1] alle verschillende soorten onheil waar ons lichaam, net als onhan-
delbare kinderen, in terecht kan komen. [all different sorts of evil/
mischief/calamity where our body, just like unruly/unmanageable
children, can end up]
[2] We knikten met onze hoofden als interpunctie voor elkaars stemmen,
om aan te geven dat ja, we er alles van weten. [We nodded with our
heads like punctuation to each other’s voices, to indicate that yes, we
know all about it]
[3] zachtjes klaagden we, onze stemmen zacht en mineur en treurig als
duiven in de dakgoottroggen. [softly complained we, our voices soft
and minor and sad like pigeons in the gutter troughs]
[4] ik hoor waar je vandaan komt, alsof de stem zelf een reiziger is, die
van een verre plaats komt. [I hear where you from come, as if the
voice itself a traveller is, who from a far place comes]
What is perhaps surprising is that this translation is error free and reads
naturally. Despite Atwood’s liking for long and interrupted sentences with
lots of commas, the Dutch sentences are idiomatic and correct and read as
natural conversation, even in places where you would expect the machine
to run into difficulty. The simple conclusion, therefore, is that such original
metaphors are not problematic for the machine since they normally require
metaphor retention, and their linguistic expression as a simile is well-suited
to direct translation.
When looking at the human translations, the students also behaved as
would be expected based on previous research into metaphor translation
(see Section 9.2) and the procedures proposed by Newmark (1988): for all
of these similes, the students opted for metaphor retention, just like Google
Translate. The official translation, however, shows some clear deviations
from this expectation. Though similes 3 and 4 have been retained, in
simile 1 ‘mischief’ has been substituted by an alternative metaphor ‘last’
[burden] (2020, p. 25). This alternate metaphor not only loses the specific
connotations of ‘mischief’ and its connection to ‘unruly children’, but is
also a highly conventional one, since ‘last hebben van’ [be troubled by] is
the common expression for health problems, unlike the ST’s description of
aches and illnesses as ‘mischief’. In simile 2, the most creative part of the
simile has been deleted: ‘We zouden elkaars stemmen met hoofdknikjes
begeleiden’ [We would accompany each other’s voices with nods] (2020,
p. 25). Though it can be argued that this partial substitution evokes
associations with music (‘begeleiden’ means ‘accompany’, as in ‘be
accompanied on the violin’), and the idea of nodding as punctuation is
perhaps not essential to the novel’s main themes, it is particularly strange
that such a creative and original image has simply been deleted.
There are also a number of variations in the human translations that
indicate where the translators struggled, but also where they may have
created added literary value. One metaphor the student translators seemed
to have struggled with is ‘I hear where you’re coming from’. Considering
how the simile develops, it makes sense to translate the metaphor directly,
yet the fact that this is not a fixed expression in Dutch may have prompted
the students to look for alternatives. In fact, they produced eight different
translations, with only one student opting for the literal ‘Ik hoor waar
je vandaan komt’ [I hear where you’re coming from], which is also the
solution the professional translator chose. The other solutions are ‘Ik snap
waar je vandaan komt’ [I understand where you’re coming from], losing
the connection with sound, as do the solutions ‘Ik zie waar je vandaan
komt’ [I see where you’re coming from], ‘Ik begrijp waar je vandaan komt’
[I understand where you’re coming from’] and ‘Ik zie wat je bedoelt’ [I
see what you mean]; the latter also deletes the ‘coming’ metaphor, thus
losing the connection with ‘traveller’. Two opt for alternative movement/
travel metaphors, namely ‘Ik kan erin komen’ [I can get into it > I can
understand] and ‘Ik ga helemaal met je mee’ [I go with you all the way >
I totally agree]; these display a certain level of creativity but are not well
matched with the continuing simile of the voice as a traveller.
One important aspect of stylistics where the human translator can
create added literary value is by retaining and recreating wordplay and
sound effects, both of which occur at different points in the excerpt,
though perhaps only noticeable to the careful reader. One instance occurs
in the simile in the opening of the chapter: ‘Above, on the white ceiling,
a relief ornament in the shape of a wreath, and in the center of it a blank
180 Aletta G. Dorst
space, plastered over, like the place in a face where the eye has been taken
out’ (1996, p. 17; emphasis added). This metaphor is in itself not difficult
to understand or translate, and both Google Translate and the human
translators (professional and student) retain the metaphor image, with only
minor variations in lexical choice between synonyms that do not affect
the metaphor image (e.g. ‘ruimte’ [space] versus ‘plek’ [spot]). Yet what
makes this metaphor creative and interesting is the sound effect created
by ‘space—place—face’ combined with the wordplay on ‘blank face’.
The reference to the meaning of ‘blank’ evoked by its collocation with
‘face’—that is, expressionless and devoid of emotion—is now projected
onto ‘space’. This elusive reference is impossible to translate directly into
Dutch since the two meanings ‘empty’ and ‘expressionless’ require two
different words.
The student translations show that they have all but one opted for the
more literal interpretation that applies to the ceiling, ‘leeg’ [empty]. Only
the choice for ‘ijlte’ counts as a creative solution here, marked and more
poetic in terms of lexical choice and providing an elusive reference to the
verb ‘ijlen’ [talk in a way that shows you are delirious]. The same student
who opted for ‘ijlte’ also found a way to recreate the sound effect, namely
by being more creative in translating ‘plastered over’ to create a rhyme with
‘face’: ‘een ijlte, met pleister gedicht, zoiets als de plek in een gezicht waar
een oog uit is gestoken’ [a hollowness, with plaster closed off, something
like the place in a face where an eye has been stabbed out]. In both
decisions, the human translator has added creativity and literariness to the
translation, without altering the metaphor. Conversely, and surprisingly,
the professional translator has altered and normalised the structure of the
sentence, combining ‘plastered over’ with ‘empty space’, thus reducing the
number of interruptions in the sentence and changing its flow.
While the creative, more literary metaphors may thus be relatively
unproblematic for both human and machine translation, it may be more
interesting to look at those cases of metaphor that may well slip through
the cracks of our attention, but which, especially in literary texts, may
actually be an inherent part of the fabric of authorial style. There are several
conventional metaphors in the excerpt. One of the least striking examples
at first glance is perhaps the use of ‘touch’ in ‘This is the kind of touch they
like: folk art, archaic, made by women, in their spare time, from things that
have no further use’ (1996, p. 17) where the noun is used in its conventional
metaphorical sense of ‘quality’ or feature’. Yet Atwood is cleverly playing
on two common collocations of ‘touch’, namely ‘a personal touch’ and ‘a
nice touch’. Normally, this kind of description of a room would include
a reference to ‘a personal touch’ which makes the omission of ‘personal’
in this context appear deliberate, reinforcing the idea that everything and
everyone in this world has been de-personalised. The formulation ‘touch
they like’ subtly suggests that this is the opposite of ‘a nice touch’; it is the
last thing the narrator likes, which is reinforced by the later confession, ‘I
hunger to touch something, other than cloth or wood. I hunger to commit
the act of touch.’ (1996, p. 21) where the unusual collocation ‘commit the
act of touch’ signals that touching is considered a crime. A quick corpus
search of the novel (see also Chapter 12, this volume) reveals that different
forms of the lemma ‘touch’ occur 42 times, creating a lexical network
throughout the novel that supports this main theme.
Looking at the translations, the most natural way to translate ‘touch’ is
by reducing it to sense, since the Dutch word ‘aanraking’ does not have the
same polysemy and Dutch readers might not understand the description of
a carpet as a form of touching. As pointed out by Guerberof-Arenas and
Toral (2022), creativity only works if it is both original and acceptable.
Retaining the metaphor may be original here, but probably not acceptable.
Google Translate of course retains the metaphor. Apparently, nothing
in the immediate context has led the algorithm to privilege a non-direct
translation (see Chapter 1, this volume). Interestingly, if we add ‘personal’
to the English sentence, the output produced by Google Translate changes
into the non-direct ‘Dit is het soort persoonlijke touch waar ze van houden’.
The human translations show that the naturalness of using ‘aanraking’
must indeed have played a role: of the eight students, only one has opted
for retention, three have used ‘detail’ [detail], one used ‘stijl’ [style] (also
the word used in the official translation), one used ‘sfeer’ [atmosphere],
one used ‘toets’ [brushstroke] and one opted for deletion of the metaphor:
‘Dit is waar ze van houden’ [This is what they like]. Though the English
metaphor is conventional and stylistically relevant, it is very hard to
translate and the preferred solution is clearly an idiomatic translation.
Equally hard to translate, for both humans and machine, is the
colloquial ‘I've heard them at it sometimes’ (1996, p. 21). Google Translate
mistranslates the idiomatic metaphor ‘to be at it’ (which conceptualises
actions and states as locations) as ‘Ik heb ze er wel eens naar horen
luisteren’ [I’ve heard them listen to it at times], which is an incredibly
complex formulation and a fluent translation but the wrong meaning.
But the machine is not the only one confused by this inconspicuous
metaphor; three of the eight students also misunderstood it and produced
incorrect translations: ‘Ik hoorde ze er soms over’ [I heard them about
it sometimes], ‘Ik hoorde ze af en toe gaan’ [I heard them go sometimes]
and ‘Ik heb ze er af en toe over gehoord’ [I have heard them about it
sometimes]. The same happens for the conventional idiomatic metaphor
‘but they found her out all right’ (1996, p. 21), which Google Translate
incorrectly translates as ‘maar ze hebben haar goed gevonden’ [they did
well in finding her]. Only one student also misunderstood the expression,
producing ‘maar ze vonden haar in prima gezondheid’ [but they found
182 Aletta G. Dorst
her in good health]. The other seven students translate the expression
correctly, but once again we see that there is much more variation in
the options they choose: ‘maar ze hadden haar door’ [but they had her
through > they were unto her], ‘maar ze hebben haar even goed te pakken
gekregen’/‘maar ze hebben haar net zo goed gepakt’ [but they caught
her all right], ‘maar ze is wel gesnapt hoor’ [but they caught her], ‘maar
ze hebben haar mooi doorzien’ [but they saw through her all right] and
‘maar ze zijn er natuurlijk achter gekomen’/‘maar ze zijn er toch achter
gekomen’ [but they came behind it of course > they discovered it all
right].
For the conventional metaphor ‘eat up’ in, ‘Jealousy, it must have
been, eating her up.’ (1996, p. 21) there is more consensus amongst
the student translators, with only one opting for the direct translation
‘opeten’ and one opting for the normal Dutch collocation with ‘verteerd
worden door’ [be digested/consumed by], and five students choosing the
near-synonym ‘opvreten’ (‘vreten’ being the vulgar form of ‘eten’, i.e.
eat without manners), which is commonly used in the Dutch expression
‘vreten aan’ for when you are plagued by negative emotions (English ‘a
gnawing guilt’). One student used a different metaphor, namely ‘vermor-
zeld door jaloezie’ [crushed by jealousy], a novel combination in Dutch.
The official translation deletes the eating metaphor, opting for ‘Ze was
natuurlijk straaljaloers’ [She was of course really jealous] (2020, p. 25),
which draws attention because of the novel compound ‘straaljaloers’.
Since ‘straal’ [beam, jet] normally combines with ‘bezopen’ [drunk] in
‘straalbezopen’, meaning ‘very drunk’, this marked lexical choice may be
intended to suggest ‘drunk with jealousy’ to the reader, creating a seman-
tic link with the following sentences, in which the women talk about a
man drinking toilet cleaner and not noticing because he was probably
too drunk.
Google Translate of course predictably retains the metaphor image:
‘Jaloezie, dat moet het geweest zijn, haar opeten’ [Jealousy, that must have
been it, eating her]. However, the interrupted sentence structure has resulted
in a grammatically incorrect (and rather ridiculous-sounding) translation,
since ‘haar opeten’ shows no agreement with the subject ‘Jaloezie’. Yet
it is not the metaphor itself which is problematic here, only Atwood’s
preference for complex and interrupted sentences. When presented with
the same metaphor in a normalized sentence structure, Google Translate
produces a translation that is idiomatic, namely ‘Paula werd dagenlang
verteerd door schuldgevoelens’ [Paula was for days consumed by feelings
of guilt]. This shows that when we analyse MT output, we have to be
aware that problems and errors in metaphor translation may not in fact
be a metaphor problem but rather a problem of linguistic realization and
sentence structure.
9.4 Conclusion
Overall, the results support Ali’s (2006) claim that both metaphor and
translation are highly complex phenomena that require considerable
knowledge of the language and cultures involved and the process of
metaphor translation cannot be reduced to a simply mechanical application
of translation procedures for pre-defined types of metaphor. Interestingly
though, and perhaps against common expectations, the results also show
that metaphor translation is not necessarily difficult or problematic for
MT engines, even in literary texts. Similar to Dagnev and Chervenkova
(2020), the current study found that creative metaphors translate quite
readily, while foregrounded metaphors and grammatical metaphors are
more problematic. This can now be argued to be the case for both humans
and machines.
While Chita and Stavrou (2020) and Dagnev and Chervenkova (2020)
found that the professional translators preserved the literary style of the
source texts very carefully, sometimes even unsuccessfully, and as such
did not show signs of Toury’s norm of standardisation, the current study
found that it was the professional translator who most often opted for
deletion and normalisation, especially for creative metaphor, similar to the
findings of Dorst (2018). Yet the substituting metaphors could often also
be related to stylistic considerations, which is in line with Park’s (2009)
claim that translators do need to keep the reader in mind and may opt
for equivalence when retaining the metaphor image disrupts the reading
experience. While humans are clearly more creative in the solutions they
consider, and their solutions in themselves may be creative and original,
this does in fact sometimes mean that the stylistic coherence of metaphor
patterns is altered, disrupted or removed.
For both humans and machines, the creative, more clearly literary
metaphors appeared to be the easiest to translate, most likely since direct
translation is simply the most suitable solution. Of course, we should
be careful not to suggest that the process leading up to these solutions
is similar since human translation involves decision-making while MT
involves computation. The machine does not reflect on the different options
in light of the text’s genre, style, skopos and the expectations of the reader.
It calculates what is the most likely translation for words in context, and
as such the decision to retain or alter a metaphor may be affected more
by sentence context, and the training data, than the metaphor’s form or
function.
When metaphors are conventional, especially in the form of fixed
collocations or idiomatic expressions, human translators show more
variation in their solutions and the machine starts making mistakes.
What makes such metaphors difficult to translate, especially for a
machine, is that they require careful attention to the way the metaphor
184 Aletta G. Dorst
is used as part of larger stylistic patterns and is meant to evoke particular

connotations and associations in the reader. Such a dynamic and careful
balancing of meaning, form and function is hard to capture in the kind of
‘hard-and-fast rules’ in current models of metaphor translation that link
specific procedures to specific types of metaphor. To improve the quality
of translation of metaphors by generic and literary MT engines, we should
not focus only on the creative and literary, but instead look more closely at
those metaphors that are so mundane that they easily escape our attention.
The least creative metaphors often require the most creative solutions,
and this is exactly where human translators can start showing off what it
means to re-create literary style.
References
Al-Hasnawi, A. R. (2007). A cognitive approach to translating metaphors.
Translation Journal, 11(3). https://translationjournal.net/journal/41metaphor
.htm
Ali, A. S. M. (2006). On the translation of metaphor: Notions and pedagogical
implications. International Journal of Arabic–English Studies, 7, 121–36. http://
www.ijaes.net/article/FullText/6?volume=7&issue=1
Alvarez, A. (1993). On translating metaphor. Meta, 38(3), 479–90. https://www
.erudit.org/fr/revues/meta/1993-v38-n3-meta341/001879ar.pdf
Atwood, M. (1996 [1985]). The Handmaid’s tale. Vintage Books.
Atwood, M. (2020 [1987]). Het verhaal van de dienstmaagd. Trans. G. de Blaauw.
Prometheus.
Besacier, L. (2014). Traduction automatisée d’une œuvre littéraire: Une étude
pilote. Traitement Automatique du Langage Naturel (TALN), Marseille.
https://hal.inria.fr/hal-01003944
https://doi.org/10.4324/9780429280641
Burmakova, E., & Marugina, N. (2014). Cognitive approach to metaphor
translation in literary discourse. The XXV annual international academic
conference, language and culture, 20–22 October 2014. Procedia—Social and
Behavioral Sciences, 154, 527–33. https://tinyurl.com/4xwwckmw
Chita, A., & Stavrou, C. (2020). The metaphor in literature and the effect on
translation. Nordic Journal of English Studies, 19(2), 117–42. https://njes
-journal.com/articles/abstract/10.35360/njes.562/
Dagnev, I., & Chervenkova, Z. (2020). Comparative study of metaphor in literary
texts and their translations. 5th International e-Conference on Studies in
Humanities and Social Sciences. http://centerprode.com/conferences/5IeCSHSS
.html
Dagut, M. (1976). Can metaphor be translated? Babel, 22(1), 21–33. https://doi
.org/10.1075/babel.22.1.05dag
Dorst, A. G. (2011). Metaphor in fiction: Language, thought, communication.
Boxpress. https://tinyurl.com/3x8fcd8d
Dorst, A. G. (2015). More or different metaphors in fiction? A quantitative cross-

register comparison. Language and Literature, 24(10), 3–22. https://journals
.sagepub.com/doi/10.1177/0963947014560486
Dorst, A. G. (2018). Translating metaphorical mind style: Machinery
and ice metaphors in Ken Kesey’s One Flew over the Cuckoo’s Nest. Perspectives,
27(6), 875–89. https://www.tandfonline.com/doi/full/10.1080/0907676X.2018
.1556707
Genzel, D., Uszkoreit, J., & Och, F. J. (2010). ‘Poetic’ statistical machine translation:
Rhyme and meter. Proceedings of the 2010 Conference on Empirical Methods
in Natural Language Processing, Cambridge, MA (pp. 158–66). https://
aclanthology.org/D10-1016/
Greene, E., Bodrumlu, T., & Knight, K. (2010). Automatic analysis of rhythmic
poetry with applications to generation and translation. Proceedings of the
Cambridge, MA (pp. 524–33). https://aclanthology.org/D10-1051/
Guerberof-Arenas, A., & Toral, A. (2022). Creativity in translation: Machine
translation as a constraint for literary texts. Translation Spaces, 11(2), 184–
212. https://doi.org/10.1075/ts.21025.gue
Jones, R., & Irvine, A. (2013). The (un)faithful machine translator. Proceedings
of the 7thWorkshop on Language Technology for Cultural Heritage, Social
Sciences, and Humanities (LaTeCH 2013), Sofia, Bulgaria (pp. 96–101).
Association for Computational Linguistics. https://aclanthology.org/W13-2713/
Mandelblit, N. (1995). The cognitive view of metaphor and its implications
for translation theory. Translation and Meaning, 3, 483–95. https://www
.sciencedirect.com/science/article/pii/S1877042814056304
Newmark, P. (1981). Approaches to translation. Pergamon.
Newmark, P. (1988). A textbook of translation. Prentice Hall International.
Nida, E. A. (1964). Towards a science of translating, with special reference to
principles and procedures involved in bible translating. Brill. https://www
.academia.edu/40285991/Toward_a_Science_of_Translating_by_EA_Nida
Park, O. (2009). The issue of metaphor in literary translation: Focusing on the
analysis of a short story translation. Journal of Language & Translation, 10(1),
155–75. https://www.researchgate.net/publication/314084692_The_Issue_of
_Metaphor_in_Literary_Translation_Focusing_on_the_Analysis_of_a_Short
_Story_Translation
Schäffner, C. (2004). Metaphor and translation: Some implications of a cognitive
approach. Journal of Pragmatics, 36, 1253–69. https://www.sciencedirect.com/
science/article/abs/pii/S0378216604000244
Schäffner, C. (2017). Metaphor in translation. In E. Semino & Z. Demjen (Eds.),
The Routledge handbook of metaphor and language (pp. 247–62). Routledge.
https://doi.org/10.4324/9781315672953
Shuttleworth, M. (2017). Studying scientific metaphor in translation: An inquiry
into cross-lingual translation practices. Routledge. https://doi.org/10.4324
/9781315678085
Steen, G. J., Dorst, A. G., Herrmann, J. B., Kaal, A. A., & Krennmayr, T. (2010).
Metaphor in usage. Cognitive Linguistics, 21(4), 765–96. https://doi.org/10
.1515/cogl.2010.024
186 Aletta G. Dorst
using SMT. Proceedings of the Fourth Workshop on Computational Linguistics
for Literature, NAACL, Denver, CO (pp. 123–32). https://aclanthology.org/
W15-0714.pdf
Van Den Broeck, R. (1981). The limits of translatability exemplified by metaphor
translation. Poetics Today, 2, 73–87. https://doi.org/10.2307/1772487
Venuti, L. (Ed.). (2021). The translation studies reader, 4th ed. Routledge. https://
doi.org/10.4324/9780429280641
Voigt, R., & Jurafsky, D. (2012). Towards a literary machine translation: The
role of referential cohesion. In D. Elson, A. Kazantseva, R. Mihalcea, & S.
Szpakowicz (Eds.), Proceedings of the NAACL-HLT 2012 Workshop on
Computational Linguistics for Literature, Montréal, Canada (pp. 18–25).
Association for Computational Linguistics. https://aclanthology.org/W12-25
.pdf
Part 3
Corpus Linguistics, Text-

Visualisation and Literary
Translation

10 KonText in Trilingual Studies—
Supporting Phraseology
Translation Based on the EPB
Corpus
Angelika Peljak-Łapińska
10.1 Introduction
Phraseology is notoriously one of the most difficult parts of translation,
as well as areas of learning. It is not only problematic to find the best
analogue in the target language, but also to identify fixed expressions in the
source language. The stakes are high, as the correct usage of phrasemes, or
fixed expressions, makes the final translation seem natural and determines
whether it is successful or not. It also influences the mutual understanding
between two speakers of a language. Therefore, translators, as well as
language learners, seek the best resources for finding the solutions they
need, and these are often parallel corpora. They are perfect for the task, as
will be illustrated below.
This chapter starts by discussing the problem of translating phraseology,
where it comes from and why it is not trivial. The following section presents
the KonText application, discussing in detail various options available for
the users, as well as the composition of the English–Polish–Belarusian (EPB)
Parallel Literary Corpus which can be queried in KonText. Finally, the
chapter focuses on the case study of make as a phrasal verb and as a core of
the phrase. The examples discussed are make up, make […] difference and
make up one’s mind. These phraseological expressions and their Polish
and Belarusian translations are used to showcase the benefits of using a
trilingual corpus and the KonText application in translating phraseology.
10.2 Phraseology in Translation
Countless articles have been devoted to the problem of translating
phraseology. Often students spend an entire semester of their Translation
Studies programme1 learning exclusively about the intricacies of
1 Such as ‘Phraseology and Translation’ for the Master’s in Translation and Interpret-
ing at UC Louvain or ‘Comparative Phraseology’ for students in Applied Linguistics at
DOI: 10.4324/9781003357391-14
190 Angelika Peljak-Łapińska
phraseology and various solutions recommended for translating fixed

expressions. There are a few aspects to consider in the process. Firstly, how
general linguistics, phraseology and Translation Studies interact. Secondly,
how phrasemes are classified and what their common characteristics are.
Thirdly, how to recognise and interpret such expressions, which is a non-
trivial task. ‘Decoding phraseology in the source text is far from easy for
translators and interpreters, all the more so as they are usually not native
speakers of the source language. Even experienced professionals sometimes
fail to detect the fixed or semi-fixed character of a source text construction’
(Colson, 2020, p. 65). Only then can one start thinking about using
phrasemes in the target text.
It is hard to overstate the importance of the phraseological aspect of
translation. As studies have proved over the years, there is a ‘correlation
between phraseological quality and overall quality’ (Rojo, 2018, p. 13).
This is because ‘phraseological units are an integral part of any language
and knowing these collocations and their adequate interpretation and
translation is the proof of proper and adequate translation of the whole
expression, fictions and pieces of art’ (Subbotina, 2013, p. 1492). A text,
perfectly translated in terms of grammatical correctness, but lacking in
those integral elements, would seem unnatural and be considered a
failure from a translational point of view. In other words, it would not
be positively received by its audience, and that is crucial both in terms of
effective communication, as well as in terms of commercial success, which
is important (if not the most important thing) in the translation business.
A question arising at this point in the discussion is how to seek
perfection in this important matter of translating phraseology. The simple
answer would be by thoroughly analysing the usage of an expression in
the source language and finding the best equivalent in the target language.
Fortunately, this work has been done numerous times by previous
generations of translators so compiling their collective effort into a corpus
and using that as a reference seems like a perfect idea. That is why it has
been implemented by so many scholars and translators who resorted to
parallel corpora for knowledge and inspiration in the process of translating
phraseology and also in the task of identifying it.
A parallel corpus is understood by most as a set of texts in language A
together with a set of their translations into one or more other languages.
An increasing number of parallel corpora are available for translators
every year, but there is one caveat to this collective resource, namely that
‘the majority of the existing parallel corpora involve English as one of the
Adam Mickiewicz University (Applied Linguistics in Poland is heavily skewed towards

translation; usually students can specialise either in one of several types of translation or
in teaching language).
KonText in Trilingual Studies 191
two languages’ (Hareide & Hofland, 2012, p. 75). Consequently, parallel

corpora are a great help when translating from or into English but less
so between other languages. This might seem a problem at first, but in
reality, because English is our lingua franca, it is much easier to compile a
trilingual corpus containing a body of texts translated between English and
language A and English and language B, than it is to find parallel texts in
languages A and B. This can be an advantage, especially when one of the
languages is a minority language. English then plays the role of a pivot, or
bridge language, as shown later on in this paper.
Such is the case with the Belarusian language. It is classified by
Ethnologue (Simons & Fenning, 2017) as a status 1 language, a statutory
national language, but with symbolic use. What this means is that
Belarusian is minoritised in all spheres of life: it is taught as a foreign
language in schools in Belarus (despite being one of the state languages), it
is not used in public television nor by politicians during public appearances,
and usually, when the citizens try to exercise their right to use Belarusian
(e.g. during a visit to the tax office or any other public institution) they
are refused the service. Even though scientific interest sustains Belarusian,
systemic governmental support does not, and thus Belarusian translators
and scholars can use only a handful of digital resources, only a few of
which allow for phraseological unit analysis. The biggest corpus of
Belarusian, the Belarusian N–corpus (BNkorpus, 2021b), gives access to
about 177 million tokens. Part of the BNkorpus is the Russian–Belarusian
Parallel Corpus with about 1.59 million tokens, and the Biblical Corpus
(BNkorpus, 2021a) containing 16 Belarusian and six non-Belarusian
translations of the Bible (no number of tokens is specified on the corpus
website). Additionally, there are multilingual corpora that contain texts
in the Belarusian language, such as the ParaSol corpus (von Waldenfels,
2011) with nearly 0.5 million Belarusian words, the Amsterdam Slavic
Parallel Aligned Corpus with 200,000 Belarusian tokens (University
of Gothenburg, 2016), and the Belarusian newspaper corpus based on
material crawled in 2011 (Deutscher Wortschatz, 2018) with almost 7.5
million tokens.
All in all, the resources available for the Belarusian language are
not extensive, but because the translation environment is becoming
increasingly multilingual, so-called small languages can use resources
dedicated to other languages to support or simply perform phraseology
translation. Firstly, investigating the source language material (e.g. with
the use of collocation analysis) gives a better idea about which expressions
are fixed and how they function. Secondly, because equivalent fixed
expressions in the target language can be sparse or even missing, an
example of translation into another familiar language provides additional
context or simply knowledge about the meaning of a particular expression
(cf. Chapter 14, this volume). Belarusian translators, when facing the lack
of a direct English–Belarusian dictionary or corpus, very often resort to
Russian–English or Polish–English resources. And recently one additional
trilingual resource has been made available, namely the English–Polish–
Belarusian Parallel Literary Corpus. The following section discusses
the composition and advantages of the EPB Corpus, not only for the
translators or learners of the minority Belarusian language, but also for
Polish.
10.3 Navigating the English–Polish–Belarusian Parallel Literary

Corpus in KonText
In a nutshell, the EPB Corpus is a collection of 20th- and 21st-century lit-
erary prose texts in English that have been translated into both Polish and
Belarusian. It contains both long (novels) and short forms (short stories,
novellas). In the case of multiple translations, only one version is chosen,
usually the first translation, otherwise the easiest to obtain. Literary works
have been chosen, rather than non-fiction, because translations exist, ena-
bling comparison, and because literary works tend to be rich in cultural
items and linguistically diverse. Literary prose provides or tries to pro-
vide examples of non-specialised uses of language, and so prose (rather
than plays or poems) was chosen for the corpus. Moreover, prose is the
most popular form translated from English into Polish and Belarusian. To
ensure the relevancy for the contemporary learner or professional user,
only contemporary, that is 20th- and 21st-century texts, were included.
The number of works is slightly over 100, mostly by male authors, and the
total size of the corpus is almost 10 million words (Peljak-Łapińska, 2020).
The numbers mentioned above relate to the raw text compiled in the cor-
pus, but what really matters for translators is the aligned corpus, that is a
corpus where a segment from the source text has been aligned with a corre-
sponding unit in the target text. The process of manually aligning the texts
is time-consuming but guarantees a higher quality of output data than in
the case of automatic alignment, especially in the case of literary works.
This is why the EPB Corpus was aligned by hand in Mantel (Pęzik, 2016).
Because of the cost in terms of time and work, only part of the corpus was
aligned, and that part is readily available in the KonText application.
KonText is ‘an advanced, highly customizable corpus query interface

built on top of core libraries of the open-source corpus search engine
NoSketch Engine (NoSkE). (...) [It] is a fully operational and mature
software deployed at the CNC [Czech National Corpus] since 2014
that currently handles thousands of user queries per day’.
(Machálek, 2020, p. 7003)
After logging into the KonText application, the translator or scholar has
access to the entirety of the EPB Corpus and also to the aligned part (over
700,000 words in total). After logging in (setting up an account is required
to fulfil the copyright requirements) at https://kontext.clarin-pl.eu, we get
access to the KonText interface with all its functions.
The default window, Search, in the corpus, has three main elements:
Corpus, Query type and Query. Query is a window for entering the search
term. Query type allows for specifying whether the search term is a spe-
cific word form, lemma, phrase, character or a query in a specific query
language (Corpus Query Language (CQL) in KonText). The Corpus but-
ton shows available corpora; only after choosing a corpus which is part
of a larger parallel corpus does a new option appear. The ‘Add a Corpus’
button displays a list of subcorpora aligned with the user’s first choice.
Choosing ‘Parallel Corpus’ results in adding the second three-element win-
dow allocated to the other corpus. This operation can then be repeated to
produce query windows for two or three aligned corpora. Additionally, the
user can narrow down the search by filtering selected word forms (option
‘Specify Context’) and by choosing texts with specific properties (option
‘Specify Query’ according to the meta-information), such as publication
date or the title of the text.
Typing the query is obligatory for the first corpus. Such a search returns
segments containing the search term (be it a character, word or phrase) in
the first selected corpus and the aligned segments from the parallel corpus
(or corpora). The search can be narrowed down by typing the query term
for the parallel corpus (or corpora) as well. In that case, only the segments
containing the query in both (or more) languages will be displayed.
KonText highlights words that were introduced in the Query window
and shows some basic statistics: overall number of hits, number of hits per
million (in relation to the whole corpus) and the average reduced frequency
(ARF), which combines the simple word count and document count, that
is the number of occurrences in each document. Clicking the highlighted
word or the segment (when no word is highlighted) opens a pop-up
window displaying a 150-word fragment of text surrounding the search
term (or the segment). By default, the results are shuffled, which means
that even if a search term appears a few times in one text the excerpts
from this particular text will not be displayed one after another but among
fragments of other texts containing the query. This is one of the ways of
protecting copyright, but it makes it more difficult to read substantial parts
of the whole text using the corpus search interface.
The search results can be enhanced by means of filtering (Filter tab).
KonText allows positive filtering, that is, selecting part of the result, and
negative filtering, namely removing some concordances according to words
(characters, phrases or any other criteria specified with the use of the query
language) occurring within a certain proximity of the main search term

(this proximity is also defined by the user).
Another option available in KonText is creating word lists (Query tab).
KonText allows for reducing the word list according to specific criteria
expressed by a regular expression, that is, by a sequence of characters
(including symbols, such as an asterisk, or marks, such as brackets). Such
a sequence very precisely specifies the search pattern and allows for a very
detailed query. The user can specify the minimum frequency of a word,
as well as add a white- and/or black-list or a list of words accepted and
excluded from the query. The resulting wordlist can be ordered according
to simple word count, document count or ARF. Details of how the ARF is
calculated are available in the Czech National Corpus manual (Pojarová,
2016). The wordlist can be saved (Save tab at the top of the window) in
either CSV, XML or TXT format. In the case of these options, only the
first 50 lines are recorded. In order to save more lines or the full result the
user needs to choose Custom in the Save tab.
KonText offers complex sorting options for the concordance lines
(Concordance tab). In the case of a high number of hits, it can create a
random sample from the results (Concordance tab), in which case the
user can define the size of such a sample. Additionally, it includes the
functions of creating a frequency distribution according to a number of
criteria (Frequency tab), and collocation search (Collocations tab). A
full description of the possibilities offered by the KonText application is
available in the user manual (Help tab).
An important element of all KonText functions is the query language,
namely CQL (mentioned above). Like all query languages, CQL (used e.g.
in SketchEngine) is based on regular expressions, expanding and adapting
them for particular needs. What is particular to CQL is that it enables
the user to specify structural attributes, such as sentence or document ID,
in addition to querying in relation to grammatical annotation. It makes
searching the corpus much more versatile, provided the user is familiar
with the underlying structure of the database.
After outlining the characteristics of the EPB Corpus and the KonText
application, the next section shows how the right resource, the trilingual
parallel corpus, and the right tool can support a translator working with
phraseology, particularly (but not exclusively) a translator of a minority
language.
10.4
Make in the EPB Corpus
As an example to illustrate the use of the EPB Corpus and the KonText
application, we shall take a close look at the verb make, as a centre of both
phrasal verbs and phraseological expressions. This choice is motivated by
the broad usage of the word in the source language and its high frequency
in the aligned corpus (393 hits in an English component of over 263,000

words).
One of the most common make phrasal verbs across the EPB Corpus is
make up. It appears in the aligned EPB corpus in six different contexts and
Table 10.1 summarises different ways it has been translated into Polish
and Belarusian.
Obviously, phrasal verbs are not always translated as phrases, but quite
the opposite. Almost all of the Polish translations are verbs; one of them—
sprawić sobie—is a reflexive verb and thus it is composed of two words.
In the case of the Belarusian language, four translations are verbs and one
is a noun.
There are two things immediately evident in the table. Firstly, there is
one context in which make up was omitted in the Belarusian translation.
This indicates that a bilingual resource, especially involving a minority
language, might be lacking in some uses of phraseological expressions, even
if they are frequent in the source language. The input from an additional
language increases the translator’s chances of finding a suitable translation
and provides inspiration for additional translation solutions regardless
of the linguistic distance between the languages. What is interesting is
that this works both ways. There is a case in which make up has been
translated more diversely into Belarusian than into Polish, which might
be an advantage for a Polish translator knowing East Slavic languages,
Belarusian in particular.
As mentioned in Section 10.3, with the use of KonText we can analyse
the collocations of a verb and find the most common phrasal expression
containing it. For make in the EPB Corpus that is make […] differ-
ence (when searching for collocations with default settings), and so this
phraseme will serve as our second example. In all occurrences, the com-
ponent difference is accompanied by a descriptor or two. To find these
Table 10.1 Make up in the EPB corpus. Back-translations in square brackets.
Polish Belarusian
make up (1) posprzątać [to tidy up] прыбраць [to clean]
make up (2) układać [to compose] прыдумляць [to come
up with]; скласці [to
compose]
make up (3) wypisywać [to write]; sporządzić none
[to draw up]
make up (4) sprawić sobie [to buy oneself зрабіць [to make]
something]
make up (5) zniechęcić [to daunt] адгаварыць [to dissuade]
make up (6) być ceną [to be a price] адплата [a payback]
occurrences together with their Polish and Belarusian translations we have

to use the option of querying the corpus with CQL. The query taking
into account possible descriptors in between the elements of the expression
looks like this: [lemma="make"][][]?[lemma="difference"]. The result of
the search is presented in Table 10.2.
The literal translation of the English expression make (a) difference
into Polish is robić różnicę. The word różnica (difference) appears in six
translations in Table 10.2, whereas the verb robić (make) only appears in
four cases. The Belarusian translations show even less consistency. The
Belarusian equivalent of make a difference is рабіць розніцу, but the noun
розніца (difference) appears in two translations, whereas the verb рабіць
(make) in only one. This is an excellent example demonstrating that a
minority language may, in many cases, exhibit a low level of phraseologi-
cal equivalence.
Phraseological equivalence (PE) can be defined as the tendency for

translators to use a target language phraseological unit, such as an
idiom, a restricted or open collocation as an equivalent of the corre-
sponding source language phraseological unit. […] a low level of phra-
seological equivalence may result in an insufficient level of idiomaticity
of the translation.
(Pęzik, 2016, p. 78)
The consequences of the aforementioned low level of idiomaticity are

twofold. First, they can impact readers who
may have to invest a larger amount of cognitive effort in understanding

it than the readers of the original. Secondly, a non-idiomatic translation
may be significantly more ambiguous than the original text, whose
meaning is constructed, to the extent that it is idiomatic, from highly
conventionalized phraseological units.
(ibid.)
Based on the material from the EPB Corpus we can deduce that Belarusian
exhibits a low level of phraseological equivalence (PE) in the case of the
expression make […] difference. A trilingual corpus gives a Belarusian
translator a double advantage. First, it makes the translator aware of that
low PE and offers a whole spectrum of translation possibilities. At the same
time proving that there is no regularity informs the translator that there is
a space for creativity. Moreover, the trilingual corpus shows increased PE
in another language (in the case of the EPB Corpus, Polish) and points to
a solution with better prospects of becoming the regular one. Similarly to
the case of the phrasal verb make up, the exercise of searching for various
Table 10.2 Make […] difference in the EPB corpus. Back-translations in square

brackets.
English Polish Belarusian

Love, true love, Miłość, prawdziwa miłość Каханне, сапраўднае
made all the stanowiła tu całą różnicę. каханне ўсё перамяняе.
difference. [Love, true love made here all[Love, true love changes
the difference.] all.]
‘Personally,’ the — Osobiście — odpowiedział —Што датычыцца мяне, я
grocer said, ‘I Sam — uważam, że to bez не бачу ў гэтым ніякай
don't believe różnicy. розніцы, — сказаў
it makes any [—Personally—responded прадавец.
difference.’ Sam—I think it makes no [—When it comes to me, I
difference.] don’t see any difference
in this,—the grocer said.]
Supposing it was Teraz przypuśćmy, że jest on Калі ж да хатніх Парсючкоў
Fierce with Pigs, groźny dla prosiaczków, ён Люты, ці істотны той
would it make to czy jest mu obojętne, факт, што прапарсюка
any difference że jeden Prosiaczek miał Парсючка звалі
if the Pig had a dziadka, który nazywał się Неўпаўнаважаным У.?
grandfather called Wstęp Bronisław? [If he is dangerous to
TRESPASSERS [Now let’s assume that he domestic Piglets, is
WILLIAM? is dangerous to piglets, important the fact, that
then would he care, that the grandpig of the Piglet
one Piglet had a grandpa was called Unauthorized
whose name was Entrance E.?]
Bronislaw?]
A little I miejcie trochę Względów, Заўважце—крышку
Consideration, a trochę Troski o Innych. W Роздуму, крышку
little Thought for tym cała rzecz. Клопату пра Іншых, і
Others, makes all [And have a little ўжо зусім інакш.
the difference. Consideration, a little [Note—a little bit of
Concern for Others. That's Thought, a little bit of
the whole thing.] Care for Others, and
it’s already completely
different.]
‘It hardly makes any — Na tym etapie nie robi — На дадзеным этапе гэта
difference at this to specjalnej różnicy — практычны нічога не
stage,’ said Ford, odpowiedział Ford. — мяняе,—адказаў Фоль
‘let them have Niech mają trochę zabawy. ксваген,—хай хлопцы
their fun.’ [At this stage it does not пазабаўляюцца.
make a special difference— [At this stage it practically
Ford answered.—Let them doesn’t change a
have some fun.] thing,—Volkswagen
answered,—let the boys
have some fun.]
(Continued )
Table 10.2 Continued
I looked up at the Spojrzałem na posąg Harriet Я паглядзеў на статую

statue of Harriet White, a ona stała Хэрыет Уайт, якая, як
White and there tam dalej, spoglądając звычайна, не адрывала
she still stood, niewzruszonym свайго каменнага позірку
looking stonily spojrzeniem na park po ад парку, быццам нічога
away towards the drugiej stronie ulicy, не адбылося, быццам
Garden across jakby nic szczególnego воля нічога не значыла
the way, as if nie zaszło, jakby taka ў гэтым жорсткім і
nothing of any determinacja na tym бессэнсоўным свеце
particular note okrutnym i twardym а магчыма, што яшчэ
had happened, świecie nie miała żadnego больш страшна, воля
as if such znaczenia... lub jeszcze была адзінай рэччу, якая
determination in gorzej, jakby była jedyną хоць нешта ў ім значыла.
a world as hard rzeczą mającą jakikolwiek [I looked at the statue of
and as senseless sens, robiącą jakąkolwiek Harriet White, who, as
as this one meant różnicę. usual, did not take her
nothing… or [I looked at the statue of stony gaze away from
worse still, that it Harriet White, and she the park, as if nothing
was perhaps the was still standing there, had happened, as if the
only thing which looking with impassive will meant nothing in
meant anything, look at the park across the this cruel and senseless
the only thing street, as if nothing special world, and perhaps even
that made any had happened, as if such more frighteningly, the
difference at all. determination in this cruel will was the only thing
and hard world meant that meant something
nothing … or worse, as in it.]
if was the only thing that
made any sense, made any
difference.]
It made no Było jej wszystko jedno. Ёй няма ніякай розніцы.
difference to her. [She didn’t care.] [It made no difference to
her.]
It might not even Być może nie robiłoby im Аднак калі б ім распавялі
have made much nawet specjalnej różnicy, пра тое, якой уладай
difference to gdyby wiedzieli, ile Прэзідэнт Галактыкі
them if they’d władzy naprawdę posiada валодае насамрэч
known exactly prezydent Galaktyki; (ніякай зусім), дык гэтае
how much power gdyby wiedzieli, że nie з'яўленне, пэўна, не
the President posiada jej w ogóle. рабіла б на іх вялікага
of the Galaxy [Perhaps it wouldn't even ўражання.
actually wielded: make much difference to [However, if they were
none at all. them if they knew how told about what kind
much power the President of power the President
of the Galaxy really had; if of the Galaxy actually
they knew he didn't have has (none at all), then
it at all.] this appearance would
probably not make a big
impression on them.]
(Continued )
Table 10.2 Continued
If it was Fond of A jeśli lubi, to czy nie robi А калі Ставіцца Прыязна,
Pigs, did it make mu różnicy, jaki to jest ці адыгрывае нейкую
any difference rodzaj prosiaczka? ролю—гэта племянны
what sort of Pig? [And if he likes, then does Парсючок ці не
is make any difference to племянны?
him, what kind of piglet [And if he Is Friendly, then
is it?] does it play any role if
it is Piglet’s relative or
not?]
phrasal expression translations might prove inspirational for translators

working with the middle language of the trio in the trilingual corpus.
Among the EPB Corpus occurrences of the aforementioned make up we
encounter one which is part of a larger, idiomatic expression, namely make
up one’s mind. The CQL query [lemma="make"][lemma="up"][]?[lemma
="mind"] returns the results compiled in Table 10.3.
In the case of translating an idiom, we see the problem of phraseological
equivalence more clearly than in the previous example. In the Polish trans-
lations, one is podjąć decyzję and three are zdecydować się, a verb derived
from the noun decyzja (Eng. decision; used in podjąć decyzję, Eng. make
a decision). In Belarusian, on the other hand, each translation choice is
different, and it is clear there is no established equivalent of the source lan-
guage expression. What the result of this experiment suggests is that the PE
increases for an idiom in a bigger, more widely used language, whereas it
decreases in a minority language. The benefit of using a trilingual resource
when working with idioms is the same as in the case of regular phraseo-
logical expressions.
The three examples from the EPB Corpus—make up, make […] differ-
ence and make up one’s mind—illustrate the strength of using a trilingual
corpus in translation work, especially for phraseology translation. Two
languages complement each other when aligned with the third one and it
is particularly evident when these languages include a lingua franca and
a minority language, as in the case of the EPB Corpus. Using a versatile
tool such as KonText, a translator can easily analyse the material, make
comparisons and find an adequate solution for a translation problem, par-
ticularly one involving fixed expressions. In the case of a minority language
like Belarusian, a translator will most likely find missing information in the
material from the more widely used language, e.g. Polish, whereas the non-
minority translator (e.g. into the Polish language) might use the minority
language as a source of translation inspiration.
Table 10.3 Make up one's mind in the EPB Corpus.

In the end, Minnie W końcu Minnie napisała У рэшце рэшт Мінні напісала
wrote to Hubert do Huberta donosząc Х'юберту, што прыняла
telling him that mu, że się zdecydowała. рашэнне.
she had made up [At last Minnie wrote to [At last Minnie wrote to
her mind. Hubert telling him that Hubert, that the made a
she made a decision.] decision.]
If that Alison finally Jeśli ta Alison zdecyduje Калі Элісан нарэшце вырашы
made up her się w końcu na rozwód па развесціся з Морысам,
mind to divorce z Morrisem, to co się што з усяго гэтага атрыма
Morris, then stanie? ецца?
how would it [If this Alison finally [If Alison at last decides to
go? decides in favour of the divorce Morris, what will
divorce with Morris, come of that?]
then what will happen?]
She had neither Nie miała przyjaciółek Яна не мела ні сябровак,
friends nor ani krewnych, którzy ні крэўных, якія маглі
relatives to zniechęciliby ją do б пераканаць яе не
talk her out of stosowania metody карыстацца метадам правіль
her belief in oddychania (choć нага дыхання (аднак, шчыра
the Breathing muszę przyznać, że кажучы, не думаю, што
Method wątpię, aby ktokolwiek знайшоўся б такі чалавек,
(although, in all zdołał zniechęcić ją do які б мог адгаварыць яе
fairness, I must czegokolwiek, skoro już ад таго, што яна палічыла
add that I doubt podjęła decyzję), gdyby небходным для сябе), калі
anyone ever przekonała się do niej. яна паверыла ў яго.
talked her out of [She had no friends or [She didn't have any friends
anything once relatives who would or relatives who could
she had made up dissuade her from using convince her not to use the
her mind on the the breathing method method of correct breathing
subject) once she (although I must admit (although, frankly, I don't
came to believe that I doubt that anyonethink there would be
in it. could dissuade her from such a person who could
doing anything once she dissuade her from what she
had made a decision) if considered necessary for
herself), when she believed
she was convinced to it.]
in it.]
But he saw on Ale po drugiej stronie Але на другім беразе сярод
the other side, zobaczył wśród drzew дрэў стаяў эўрапейскага
nestling among dom białego człowieka. тыпу будынак. Гэта
the trees, a white Zdecydował się szybko прымусіла яго адважыцца
man's house; i wstąpił ostrożnie na асцярожна ступіць на
he made up his most. першае бервяно.
mind and, rather [But on the other side he [But on the other side, among
gingerly, began saw a white man's house the trees, there was a
to walk. among the trees. He European-style building.
made a decision quickly This made him dare to step
and stepped cautiously carefully on the first log.]
onto the bridge.]
10.5 Conclusions
The trilingual EPB Corpus gives translators real-life, ready-to-use solu-
tions, not only in the minority, Belarusian language but also in Polish.
Such solutions are especially important in phraseology translation because
translating phraseology is crucial for target text quality. A translator
equipped with a versatile tool such as KonText can efficiently analyse the
available material and use it to firstly identify, and secondly make well-
informed decisions concerning the translation of fixed expressions in the
source language.
When a minority language is involved, even a small bilingual parallel
corpus is a huge help for the translator, but multilingual settings provide
even more solutions. The material in another language provides addi-
tional information about the context of certain words and expressions.
Moreover, when the translation in a minority language is omitted, the
example in the other language might serve as the primary source of infor-
mation, as shown above with the example of the phrasal verb make up
in its various contexts.
Based on the examples provided in this study, it seems that minority lan-
guages have lower levels of phraseological equivalence, and consequently
they often contain more diverse options for translating fixed expressions.
Therefore, a trilingual corpus can also be a source of inspiration for non-
minority translators, as is shown by the examples of Belarusian and Polish
translations of the English expressions make […] difference and make up
one’s mind. Additionally, such a resource is an advantage for language
learners.
References
BNkorpus. (2021a). Biblijny korpus [Biblical corpus]. http://biblija.bnkorpus.info
/index.html
BNkorpus. (2021b). Biełaruski N–korpus [Belarusian N–corpus]. http://bnkorpus
.info/
Colson, J.-P. (2020). Computational phraseology and translation studies: From
theoretical hypotheses to practical tools. In J.-P. Colson & G. Corpas Pastor
(Eds.), Computational phraseology (pp. 66–81). John Benjamins. https://doi.org
/10.1075/ivitra.24.04col
Deutscher Wortschatz. (2018). Leipzig Corpora Collection—Wortschatz
Belarusian. http://corpora.uni-leipzig.de/de?corpusId=bel_newscrawl_2011
Hareide, L., & Hofland, K. (2012). Compiling a Norwegian–Spanish parallel
corpus: Methods and challenges. In M. P. Oakes & M. Ji (Eds.), Quantitative
methods in corpus-based translation studies, 51 (pp. 75–114). John Benjamins.
https://doi.org/10.1075/scl.51.04har
Machálek, T. (2020). KonText: Advanced and flexible corpus query interface. In
Proceedings of the Twelfth Language Resources and Evaluation Conference,
Marseilles, France (pp. 7003–8). European Language Resources Association.

https://aclanthology.org/2020.lrec-1.865/
Peljak-Łapińska, A. (2020). Design, compilation and applications of an English–
Polish–Belarusian parallel literary corpus [Unpublished PhD dissertation].
Swansea University. https://doi.org/10.23889/SUthesis.62708
Pęzik, P. (2016). Exploring phraseological equivalence with Paralela. In E.
Gruszczyńska & A. Leńko-Szymańska (Eds.), Polish-language parallel corpora
(Vol. 1, pp. 67–81). University of Warsaw, Faculty of Applied Linguistics,
Institute of Applied Linguistics. http://rownolegle.blog.ils.uw.edu.pl/files/2016
/03/04_P%C4%99zik.pdf
Pojarová, V. (2016). ARF (average reduced frequency). https://wiki.korpus.cz/
doku.php/en:pojmy:arf
Rojo, J. L. (2018). Phraseology as indicator for translation quality assessment
of museum texts: A corpus-based analysis. Cogent Arts & Humanities, 5(1).
https://doi.org/10.1080/23311983.2018.1442116
Simons, G. F., & Fenning, C. D. (Eds.). (2017). Ethnologue: Languages of Africa
and Europe, Twentieth Edition. SIL International.
Subbotina, V. (2013). Challenges of translating phraseological units. Procedia—
Social and Behavioral Sciences, 70, 1487–92. https://doi.org/10.1016/j.sbspro
.2013.01.216
University of Gothenburg. (2016). ASPAC—Swedish–Belarussian corpus.
Språkbanken (the Swedish Language Bank). https://spraakbanken.gu.se/eng/
resource/aspacsvbe
Von Waldenfels, R. (2011). ParaSol—A parallel corpus of Slavic and other
languages. http://parasolcorpus.org
11 Voyant Tools’ Little Outing
How a Text Reading and Analysis
Environment Can Help Literary Translators
Lisa Horenberg
11.1 Introduction
To explore the opportunities for non-automated translation technologies
in literary translation, as outlined in the introductory chapter to this vol-
ume, this chapter focuses on technological innovations for literary source-
text analysis (STA) in particular. Inspired by traditional definitions of
computer-assisted translation (CAT) tools, this chapter provides insight
into the benefits of corpus linguistics in the preliminary, STA phase of the
literary translation process. More specifically, it tests the hypothesis that
text-analysis software Voyant Tools (Sinclair & Rockwell, 2022; hence-
forth, ‘VT’) can assist literary translators by providing them with basic yet
valuable information about their source text (ST), which might help them
achieve a more detailed understanding of the ST and start their translation
process more efficiently.
To identify possible benefits for literary translators using VT for STA,
this chapter focuses on a workflow-oriented case study1 in which VT was
used to analyse Evelyn Waugh’s (1903–1966) short story Mr Loveday’s
Little Outing (1951, first published in 1936; henceforth, ‘MLLO’). VT is a
free, online, and open-source text-analysis program that comprises a range
of visualisations to support distant reading (see Section 11.2). ‘Designed to
facilitate reading and interpretive practices for digital humanities students
and scholars as well as for the general public’,2 the tool efficiently provides
quantitative data about one or more texts. Its interface (referred to as a
‘skin’ by VT) is easy to use and requires little experience. The user copies
and pastes or uploads their text or document on the home page, and VT
calculates word frequencies, correlations and other types of quantifiable
textual data. This data is then presented in five visualisation panes that
1 This case study was first carried out for my (2019) master’s thesis, which also involved
Sketch Engine and Trados Studio, and a computer-assisted (re)translation of the ST.
2 https://voyant-tools.org/docs/#!/guide/about.
DOI: 10.4324/9781003357391-15
204 Lisa Horenberg
show varied interactive visualisations (referred to as ‘tools’ by VT) which

users can navigate by selecting words they find interesting.3
This chapter first focuses on the categorisation of translation
technologies and how these might include digital reading and analysis
techniques (Section 11.2), followed by a contextualisation of the case
study to introduce the ST and methodology (Section 11.3). The case study
is presented in Section 11.4, highlighting key aspects of first traditional
and then computer-assisted STA. The discussion in Section 11.5 reflects on
and compares both analyses and VT’s role, taking into account practical
implications and comments on the study’s limitations. The conclusion
follows in Section 11.6.
11.2 Text-Analysis Tools as Translation Technology

In addition to the parameters of translation technology and CALT as set
out by this volume’s editors in their introductory chapter, this chapter dis-
tinguishes CAT tools from Machine Translation (MT) to focus on assist-
ing translators in their job rather than potentially replacing them (see also
Chapters 5 and 14, this volume).4 This perspective opens doors for research
into new technologies that might be able to assist translators in one way
or another. This includes visualisation techniques for text analysis. Taken
from neighbouring fields of Translation Studies, more specifically Digital
Humanities and Computer Science, these techniques can facilitate ‘distant’
reading5 as an alternative to traditional close reading (Jänicke et al., 2015).
Youdale (2020) innovatively combines these techniques into a close and
distant reading (CDR) model for the translation of literary style that can
be applied in four distinguished parts of the translation process (see also
Section 11.3.2).
However, visualisation tools for text analysis are not generally
considered translation technologies. As described by Alcina (2008, p. 96),
translation technology traditionally consists of five subcategories:
3 This chapter discusses some but not nearly all of VT’s functionalities, nor does it cover how
each of the tool’s features work. For more specific information on VT, see www.voyant
-tools.org/docs/.
4 Although the modern reality of Kenny’s (1999, p. 67) observation that ‘in MT the machine
seeks to replace the translator by fully automating the translation process’ is more nuanced
(see Chapter 1 of this volume), being made expendable by MT remains a genuine fear
among translators (Daems, 2021) and cases in which MT approaches human-quality trans-
lation exist (e.g. Popel et. al, 2020).
5 Coined by Moretti (2000), distant reading essentially entails applying computational
approaches (including corpus linguistics) to texts.
Voyant Tools’ Little Outing 205
1. The translator’s computer equipment

2. Communication and documentation tools
3. Text editing and desktop publishing
4. Language tools and resources
5. Translation tools
MT and CAT tools both generally reside in categories four and five; Alcina
(2008, p. 98) explains that language tools and resources are ‘designed for
the collection and organisation of linguistic data’ and mainly consist of
‘electronic dictionaries, databases and text corpora’, and that translation
tools ‘work with at least one [ST] and one [Target Text (TT)] at the same
time and establish relationships between both texts on a segment level or
on the whole text level’ (ibid.). Using text-analysis tools as translation
technologies surpasses Alcina’s categorisation: while some text-analysis
tools allow users to create bilingual corpora and/or help organise linguistic
data, they do not completely fit the categories CAT technologies typically
inhabit. Instead, text-analysis tools can logically be seen as a new, sixth
subcategory of translation technology, which better represents their added
value to the field of translation.
11.3 Contextualisation of the Case Study

11.3.1 Selection and Summary
In this case, the primary selection criteria for the ST were that its length
would allow for analyses and (re)translation into Dutch of the full text
within the scope of the project, that the text had to have been written
by an established author and that its existing Dutch translation was due
for an update. The selected ST was Evelyn Waugh’s story Mr Loveday’s
Little Outing. Waugh was an English author mainly known for the novel
Brideshead Revisited (1945), whose earlier works include Vile Bodies
(1930) and A Handful of Dust (1934). Described as ‘the funniest man of
his generation’ (A. Waugh, quoted in Wykes, 2016, p. 1) and someone
who ‘liked things to go wrong’ (A. Fleming, quoted in ibid.), Waugh had
a tendency to ‘make fun of situations usually regarded as tragic’ (Bucaria,
2008, pp. 218–19) that resulted in elegantly written, dark-humorous
stories.
To summarise, Mr Loveday’s Little Outing is set in early 20th-century
Britain and consists of three numbered parts that succeed each other quite
seamlessly. The story follows Angela Moping, a young lady who for the
first time joins her mother (Lady Moping) on a visit to her mentally ill
father (Lord Moping) in an asylum. During Angela’s visit, she meets her
father’s officious secretary and caretaker, Mr Loveday. Angela learns that
Loveday is also an inmate and has been for 35 years after he killed a young
206 Lisa Horenberg
woman. Convinced that his time at the asylum has restored Loveday’s
mental sanity, Angela arranges for him to be released. At the end of the
story Loveday voluntarily returns to the asylum, just a few hours later.
11.3.2 Methodology
Christensen et al. (2017, p. 14) explain that existing research into the
technical aspects of translation technology usually highlights the tools’
functions and that existing workflow-oriented and industrial research
often focuses ‘more on the implementation of technology in the language
industry than on the impact of this on translation processes’, or how
the implementation of technology ‘affects [translators’] minds and work
processes’ (ibid.). This case study combines the research categories of
Christensen et al. by incorporating aspects of technology-oriented research
into a primarily workflow-oriented method that is focused on the practical
use and functionality of computer-assisted STA.
The case study is inspired by and gratefully borrows from Youdale’s
aforementioned CDR model. The ST is first analysed without special
technology to highlight textual features that are deemed important or
can cause translation problems, to which the subsequent computer-
assisted STA can be compared. This first traditional type of STA is usually
an explicit step in the process for translation students but is ‘peculiarly
internalised’ in more experienced translators as ‘experience makes [them]
feel when special attention is necessary’ (Pym, 2011, p. 2).
The second part contains a freestanding6 two-step computer-assisted
STA7 that first looks at VT’s basic results and then explores them in more
detail. This method covers the first two parts of the translation process
identified in Youdale’s (2020, p. 41) CDR model, respectively ‘making
notes during and after initial reading’, and ‘preparing to translate’. By
following VT’s layout, the computer-assisted STA is expected to include a
set of elements that could be stylistically relevant on either a narratological,
lexical, or grammatical level, or on the level of context and cohesion, e.g.
‘plot structure’, ‘character names, attributes, dialogues, registers, etc.’,
6 My experience analysing the text for Part One (and translating it without CAT, which was
part of the original research) of the case study was likely to interfere with the computer-
assisted STA in Part Two to some degree. This interference was minimised by scheduling a
gap of nearly two months between working on Parts One and Two, as well as by perform-
ing the computer-assisted STA as objectively as possible. Limitations of the research are
also discussed in Section 11.4.
7 Although VT can process PDF documents, the PDF ST was converted to Word and cleaned
of formatting and special characters to increase the reliability of results before being
uploaded to VT. This process is mentioned in the original research project (Horenberg,
2019).
‘unusual words or collocations,’ ‘idiomatic expressions’, ‘sentence length

and type’ and ‘repetition of words or phrases’ (ibid.).
11.4 Case Study
11.4.1 Traditional STA
This section provides a brief STA by commenting on themes in the text and
how they are reflected stylistically. Considering the ST’s translation into
Dutch, it discusses translation issues on both the micro and macro level
and comments on what approach can best be taken for their translation.
Mr Loveday’s Little Outing represents Waugh’s iconic style through its
dark-humorous, rather ironic tone and content. With regard to content,
one of the main features that can be identified throughout the story is
Angela’s personal development. In the first part of the story when Angela
visits her father at the asylum, she is rather passive. However, this part
ends with Angela stating that Loveday’s situation ‘doesn’t seem fair’
(MLLO, p. 11) as ‘[she has] never seen anyone saner’ (MLLO, p. 10). This
is a stepping-stone to the story’s second part, which provides Angela with
more agency as it elaborates on her surrogate feeling of injustice and her
determination to help Loveday when he explains to her that he would like
to have just one little outing. In the third and final part of the story, Angela
truly becomes ‘a woman with a cause’ (MLLO, p. 13) as she works for
and achieves Loveday’s release, which in turn can be seen as a big personal
accomplishment for her. However, events turn dark when it is revealed,
albeit implicitly, that he has killed another young woman who passed him
on the road. Angela’s development is the leitmotif of the story, and the plot
twist affects her as well as Loveday, as it negates her development.
Angela’s personal development is also reflected stylistically, as her
passive role in the first part of the short story is emphasised by descriptions
in a passive voice, such as: ‘it was being made at her own suggestion’
(MLLO, p. 7). Angela is more assertive in the second part of the story,
and her thoughts and feelings are voiced more actively and clearly. This
continues in the story’s third part, where she has ‘a new purpose in life’
(MLLO, p. 13) and finally achieves something real by herself. Stylistically,
Angela’s newfound attitude and drive are represented in the flow of the
text, with its speed increasing as the narrative proceeds. These stylistic
features together reinforce Angela’s role in the story and build up suspense
for the plot twist that concludes the story.
Similarly, the story shows a pattern of confusion and uncertainty in
both style and content. A first example of this is the sudden shift from
clear, sunny skies to black squalls at a garden party in the beginning of
the story. The changing of the weather is described in phrases that are
separated by semicolons, providing a rather chaotic paragraph that mirrors
208 Lisa Horenberg
the ongoing chaos stylistically. Secondly, there are many factors that create
a certain level of unrest in the reader, including a general confusion and
inconsistency with regard to Loveday and his role at the asylum as well
as in the story itself. In these and other instances, Waugh shifts between
dialogues and background information, between direct and indirect speech
and the story’s omniscient narrator frequently changes perspective. These
shifts unsettle the reader throughout the text and foreshadow the coming
plot twist.
The plot-twist at the end of the story that negates Angela’s development
by revealing Loveday’s murderous relapse is built up to and foreshadowed
throughout the text in both style and content. These stylistic and narrative
turns are reinforced by ironic undertones that define Waugh’s text. These
features should therefore ideally all be maintained in translation. The
translation strategy thus requires the translator to bear in mind the macro-
level of the text when it comes to specific translation issues, as it is precisely
the way different elements are intertwined that characterises this text.
11.4.2 Computer-Assisted STA—Part One
This section discusses VT’s most relevant initial results as shown in

Figure 11.1 from top-left to bottom-right. Here, the STA follows the data-
driven and open-ended perspective that VT fosters, providing a general
textual analysis that is not yet translation-specific.
Before doing so, it must be noted that a list of so-called ‘stopwords’,
a Stoplist, is excluded from these results by default. There are standard
yet editable Stoplists available for a number of languages, and for English
the list mainly consists of pronouns, frequently used auxiliary verbs,
prepositions, determiners and numbers and special characters, or ‘function
words that [do not] carry as much meaning’.8 Also, throughout VT, all
words are standardised by removing capitalisation. These functionalities
and some of the basic findings introduced in this section are discussed
more thoroughly in Section 11.4.3.
First, the Cirrus in the top-left of the interface is a word cloud based
on term frequency. From this view it can be deduced that ‘angela’ and
‘loveday’ are key words in the story. Additionally, the words ‘time’,
‘moping’, ‘said’, ‘mr’, ‘day’, ‘lady’, ‘little’, ‘don’t’ and ‘doctor’ also occur
frequently. Within this pane, Cirrus can be replaced by views called Terms
and Links, each differently visualising the same frequent words and their
absolute or relative frequencies.
VT’s interactive interface functions in such a way that any word a user
selects in one pane also causes the visualisations in most other panes to
8 https://voyant-tools.org/docs/#!/guide/stopwords.
Figure 11.1 Initial results of the case study’s computer-assisted STA in Voyant Tools. https://voyant-tools.org/docs/#!/guide/about.
210 Lisa Horenberg
focus on that specific word as well. For example, selecting ‘angela’ in Cirrus
causes the standard top-centre pane, Reader, to show the distribution of
‘angela’ throughout the text. By default, Reader shows the full text and
can be navigated by scrolling down or selecting words. Reader can also
be switched to TermsBerry, another word cloud that shows how words
‘co-occur (that is, to what extend [sic] they appear in proximity with one
another)’.9 TermsBerry reveals that ‘angela’ has the most collocates with
other frequent words. Interestingly, this is not the case for ‘loveday’, but
it is for ‘don’t’, which initially seems a less significant word with a total of
only eight occurrences in the whole text. ‘Said’ shows the highest number
of the same collocates with ‘angela’, ‘doctor’, ‘lord’ and ‘father’. Not much
can be derived from these isolated results just yet, although there seems to
be some sort of dialogue between or about Angela, Lord Moping and the
doctor. When ‘said’ is selected, Reader additionally shows that this word
occurs most often in a section just before the middle of the text, indicating
that some dialogue might take place in that part of the text. These views
allow for some cautious statements about the contents and structure of
the text but require additional information from other views to yield more
telling results.
The top-right Trends view in Figure 11.1 provides some additional
insight. Trends by default shows graphs of the five most frequent words,
with a relative frequency (number of occurrences per million words) on
the vertical axis and ten default document segments on the horizontal axis
(each one-tenth of the text’s total length in words), which allow for visual
comparison of relative word frequencies. Once again, more details are
provided by the alternative top-right pane called Document Terms, which
shows absolute values of relative word frequencies in a table containing
dozens of different words. Trends provides particularly interesting
information in this case. A focus on ‘said’ seems to confirm there is indeed
dialogue just before the middle of the text and some more at about three-
quarters of the way through. When only the two most frequently recurring
terms are selected in Trends, something interesting happens as well, as
the relative frequencies for ‘angela’ and ‘loveday’ seem to correlate to a
certain extent. Both names barely occur in the first two text segments,
but then occur more and more frequently, although exact numbers vary
between segments. Considering both terms are names of characters,
logically, they would indicate some sort of character relevance to the
story. The increasing mentions of Angela point towards her increasing role
in the story. The number of occurrences of Loveday, however, starts to
decrease after his initial introduction halfway through the story. It is clear
9 https://voyant-tools.org/docs/#!/guide/termsberry.
that ‘angela’ is barely mentioned in the final quarter of the story and that,
contrary to the direction of the graph in the first eight segments, ‘loveday’
suddenly emerges again in segments nine and ten. This shift alerts users
that something is happening. Similarly, Trends shows that ‘moping’ is
mentioned quite often in the first parts of the story, but is then gradually
mentioned less frequently, indicating Lord and Lady Moping’s roles might
decrease as the story develops.
Moving on, VT’s default interface offers two more panes that each
contain various views. In the bottom-left pane, Summary initially provides
a brief summary of basic textual data (Figure 11.1). There is one single
document uploaded to VT, which contains 2,655 words with 912 unique
word forms. It also shows vocabulary density (the number of unique
words divided by the total word count), in this case 0.342, and the average
number of words per sentence, in this case 13.7. In this pane’s Documents
view, more documents can be added and the documents that are used can
be downloaded. The third view that can be used in this pane is Phrases,
which by default shows recurring phrases that are between two and
30 words long, but these criteria can be customised. The phrases in the
current default selection state there are a number of four-word phrases
that occur twice and a few shorter phrases that occur more frequently. To
learn more about the role of these repeated phrases in the ST and to be able
to decide if consistency is desired in the TT as well, this information needs
a closer look (see Section 11.4.3).
In the bottom-right pane, Contexts provides textual context the most
systematically by showing all occurrences of a word or phrase and the
words that precede and follow it (commonly referred to as Key Word in
Context (KWIC) concordance). Users can customise this view by specifying
how many words they would like to see as context both left and right of
the selected word or phrase, which is set to five by default. This pane
can also present Bubblelines, again visualising the distribution of frequent
terms across the text, whereby the text is represented as a horizontal line
and terms are represented by circles (bubbles) placed on that line. Finally,
this bottom right pane can alternatively show Correlations, a table that
is based on a ‘correlation coefficient [that] is calculated by comparing the
relative frequencies of terms (relative to each document for the corpus or
relative to each segment of document)’.10 For the current case study, these
three views confirm there are some repeating words and phrases. However,
the size of the text seems to limit the number of repetitions and respective
context which affects correlation results, indicating that the text is too
short to collect relevant data using the correlation coefficient.
10 https://voyant-tools.org/docs/#!/guide/correlations.
212 Lisa Horenberg
11.4.3 Computer-Assisted STA—Part Two
These initial results in VT hint at textual features and some of the tool’s
functionalities that call for a closer look. Firstly, as previously mentioned,
VT uses a default Stoplist to filter out words that would typically distort
results, but this list can be customised. The list not only contains words
but also numbers, punctuation, abbreviations and individual letters. This
list of Stopwords may include words that are relevant to the analysis after
all. Whereas words like ‘the’, ‘to’, ‘and’, ‘you’, ‘of’ and ‘he’ might indeed
generally not reveal much about a text, frequent use of words such as
‘whereas’, ‘however’, ‘moreover’, ‘rather’ and ‘whom’ may be stylistically
relevant as they could attest to a certain register (see Chapter 3, this volume,
which demonstrates exactly this for the German language).
Including all words by deactivating the Stoplist puts results in a different
perspective (Figure 11.2). Comparing frequent words with and without
using the Stoplist can cause words that are normally included in the
Stoplist—and so do not show up in initial results—to stand out. Although
most of these words, such as ‘a’, ‘of’, ‘to’, ‘the’ and ‘and’ are irrelevant in
this case, ‘you’ and ‘i’, which occur 42 and 48 times, respectively, are not.
Trends shows two clear peaks for ‘i’ and one peak for ‘you’ (Figure 11.3).
The peak for ‘you’ coincides with the first one for ‘i’ and corresponds
to the dialogue that occurs at the end of the first half of the story. This
is confirmed in the Reader, which also indicates that the second peak of
‘i’ occurs when Loveday answers Angela’s questions. Waugh’s use of ‘i’
and ‘you’ therefore mirrors and draws attention to the fact that Loveday
barely talks about himself in the first part of the story, where he is more
concerned about Lord Moping and Angela, but that the focus in the later
part of the story shifts towards Loveday himself.
In the case of ‘you’ and ‘i’, the required additional context is provided
by other panes. However, isolating words from their context can also be
beneficial, and the removal of capitalisation allows for more objective
interpretations or analyses. Words that are normally capitalised now do
not draw more attention than those that never are, and isolating words
from their context may furthermore invite users to contemplate the
underlying meaning behind them and reconsider their role in the text. For
example, ‘angel’ and ‘love’ stand out more prominently as parts of the
names ‘angela’ and ‘loveday’. This emphasises Angela’s angelic aspect,
referring to her role in the story in the way she steps up and cares for
Loveday’s fate. Similarly, the meaning of ‘loveday’ adds to the irony of the
story given the day of Loveday’s release can be seen as ‘the settlement of
a dispute’ (OED, n.d.) (i.e. the initially friendly rectification of him having
lived as an inmate for too long). These underlying layers cause translation
issues because the meaning behind these names is less obvious if the names
are maintained in the TT. However, attempting to find target language
Figure 11.2 Results of computer-assisted STA in Voyant Tools, Stoplist deactivated. https://voyant-tools.org/docs/#!/guide/about.
214 Lisa Horenberg
Figure 11.3 T
rends showing ‘you’ and ‘i’. https://voyant-tools.org/docs/#!/guide/
about.
equivalents can also have undesirable effects as they would domesticate the
story, while it can be argued its English setting is quintessential to the text.
The recurring use of ‘time’, ‘day’ and ‘little’ also requires a closer look.
First, the frequent use of ‘day’ and ‘time’ is interesting given both terms
contain direct or indirect references to times in the past and/or the general
passing of time, which may hint at a central theme in the story. However,
a Context search for ‘time’ and ‘day’ shows little correlation between
the occurrences of the words, nor does Trends show clear similarities or
contrasts between their distribution. Nonetheless, in a more general sense
and with regard to the theme of time, the words likely both refer to the
time Loveday has spent at the asylum and the day of his release. Similarly,
the frequent recurrence of ‘little’ is of course interesting due to the title of
the story. Trends shows the term is used regularly throughout the text, and
Context shows the term is used mostly in an ironic or derogatory sense,
for example: ‘there is a very nice little flower garden’, ‘nasty little river’,
and ‘quite a little fortune’ or ‘on our little visit yesterday’. Whereas the
repeating of ‘time’ and ‘day’ refers to the story’s theme, the repeating of
‘little’ contributes to a clear voice. All three words are likely to be deliberate
stylistic features of the text and build up the story, and it can therefore be
argued that consistency in the translation is desirable in this case.
As mentioned earlier, VT not only presents recurring individual words
but also recurring phrases. In this case, there are short phrases of two
words that occur quite frequently, such as ‘mr loveday,’ ‘lady moping’
and ‘he is’. The longest recurring phrases consist of four words, and all
occur twice and seem more significant to the story. Selecting these phrases
then shows users where and how they are used in the text as the phrase
is highlighted in the other panes. This shows all longer repeated phrases
happen in close proximity to each other and even appear within the same
linear segment (e.g. ‘i do feel that’), whereas shorter phrases may occur
throughout the text (e.g. ‘a great deal’). Although these repetitions are
subtle, their close distributional proximity to each other allows for them to
be argued to stylistically foreshadow Loveday repeating himself as well and
committing a second murder, which is why it is most sensible to maintain
these recurring phrases consistently in the translation. Although only
single words and short phrases are repeated in this case, and longer texts
might yield more compelling results, identifying repetitions is important
for translators to make conscious and well-informed decisions about them
as possible stylistic features and how this affects their translation strategy.
11.5 Discussion
With VT’s initial, basic results, the user quickly gets a sense of their text
simply by looking into frequent words and their distribution, which this
case study shows can already reveal quite a bit about a story (i.e. main
characters and themes, sections with direct speech and the plot-twist). VT’s
interface invites users to further explore their data, and having a closer look
at textual features as well as some of the tool’s functionalities (and how
these affect the results) can add nuance to otherwise standardised results.
In this way VT can help translators in the early stage of STA by providing
a general data overview of the quantified text, recognising frequent words
and giving users an idea of the structure of the text and its possible patterns.
The tool guides users to look at their text from a data-driven perspective
that allows them to easily investigate aspects they find interesting, which
can lead to more objective or even completely new insights.
In this case, the qualitative and quantitative approaches of traditional
and computer-assisted STA complement each other well because both
analyses bring forward points of attention that are not immediately picked
up in the other. The traditional STA identifies stylistic representations
216 Lisa Horenberg
of character progression and comments on general patterns, themes and

undertones that set a distinct mood, whereas the computer-assisted STA
identifies keywords, trends and word patterns. As these features can mark
the same macro-level patterns and themes identified in the traditional STA,
and because it takes minimal time and effort to access such potentially
valuable information during computer-assisted STA, VT makes a valuable
and efficient addition to a literary translator’s workflow.
Although the implementation of VT may not drastically increase
translators’ efficiency, literary translators can only benefit from being
notified of certain linguistic aspects of a text early on in their process.
Whereas the story’s title indicates it is about Mr Loveday, a translator
would normally read the entire text to learn how the story progresses and
might then have to reread the text to determine the role of stylistic features.
This is not always possible for jobs with longer texts and tighter deadlines.
In such cases, accessing this data and gaining insight into word frequencies,
repetitions and distribution provides translators with relevant information
about the ST before they start. Using VT for STA can confirm or sharpen
translators’ awareness of certain linguistic or stylistic aspects of a text that
might otherwise have gone unnoticed.
My experience in this case study shows that VT works particularly well
because of its visualisations and their interactive nature, with all views
showing different visualisations that collectively and efficiently provide
the user with data that helps them look further into the information they
find relevant, after which it remains up to them to determine whether
and to what extent results should affect translation. VT’s simple default
interface gives users insight into different aspects of how words and
phrases are used in the text and in relation to each other. Since VT
offers both basic and elaborate information on a visualisation’s specific
functionalities and options, no prior experience or knowledge is required
to operate the tool.
The case study presented in this chapter has some limitations yet shows
VT’s potential even beyond the scope of this research. As noted above, the
original research project had measures in place to ensure results could be
represented as objectively as possible. Realistically, however, they were not
completely standardised as I did all analyses and translations myself. Due
to the scope of the study, it was not possible to apply VT to a text that
had not yet been analysed and/or translated, nor was it feasible to have
multiple translators use the tool. A factor that may well have restricted
the outcomes of this study was the length of the ST. Although this was
an important selection criterion that facilitated the research as a whole,
functionalities that cover word combinations (most notably repeating
phrases and identifying collocates) could not be used to the fullest and
seem likely to yield more telling results from larger datasets.
This simultaneously highlights VT’s potential for larger projects. The

tool can process large collections at one time or can be used to analyse
books or book chapters individually, enabling users to identify differences
or similarities within and between novels or chapters. Analysing longer
texts could unlock VT’s potential to uncover even more single-word or
phrase patterns. Additionally, VT offers many more types of visualisations
than are discussed here and has plenty of opportunities for customisation.
All views in the interface can be switched around so that users can combine
views in whatever way suits them.
11.6 Conclusion
Although not all of VT’s functions are fully employed due to the limitations
and scope of this study, the tool’s varied representations of quantified
textual data give significant insight into the ST. Other benefits include the
fact that VT allows users to explore their text with minimal effort, pointing
out recurring words and phrases that might indicate themes and patterns in
the text, yet offers varied opportunities for more thorough investigations.
As such, the basic but relevant information VT gathers provides a useful
starting point for a translator to explore results and apply their findings to
their translation at their discretion.
VT demonstrates its potential to yield more sophisticated results with
regard to recurring phrases or sentences and other patterns throughout
one or more (longer) texts. Although this chapter covers only the
application of the tool to the preliminary stage of the translation process,
the original research also reflects on using VT’s findings as a reference
during translation and employs two other tools as well. Further research
might elaborate on such literary applications of text-analysis tools and
other types of translation technology. This chapter not only showcases
VT’s benefits, but it also substantiates the expansion of Alcina’s
categorisation of translation technology to include text-analysis tools as
a sixth subcategory and underlines the general relevance of computer-
assisted STA.
References
Alcina, A. (2008). Translation technologies: Scope, tools and resources. Target,
20(1), 79–102. https://doi.org/10.1075/target.20.1.05alc
Bucaria, C. (2008). Dubbing dark humour: A case study in audiovisual translation.
Lodz Papers in Pragmatics, 4(2), 215–40. https://www.degruyter.com/document
/doi/10.2478/v10016-008-0014-2/html?lang=en
Christensen, T.P., Flanagan, M., & Schjoldager, A. (2017). Mapping translation
technology research in translation studies. An introduction to the thematic
section. Hermes, 56, 7–20. https://tidsskrift.dk/her/article/view/97199
218 Lisa Horenberg
Daems, J. (2021). Wat denken literaire vertalers echt over technologie? Filter. https://
www . tijdschrift -filter .nl /webfilter /dossier /literair -vertalen-en-technologie /
januari-2021/wat-denken-literaire-vertalers-echt-over-technologie/
Horenberg, L.W. (2019). Using CAT in literary translation—How tools may
support translators in source-text analysis, translation and retranslation: A case
study of ‘Mr Loveday’s little outing’ [Unpublished Master’s thesis]. Utrecht
University. https://studenttheses.uu.nl/handle/20.500.12932/33956
Jänicke, S., Franzini, G., Cheema, M.F., & Scheuermann, G. (2015). On close and
distant reading in digital humanities: A survey and future challenges. EuroVis
(STARs), 83–103. http://diglib.eg.org/handle/10.2312/eurovisstar.20151113
.083-103
Kenny, D. (1999) CAT tools in an academic environment: What are they good for?
Target, 11(1), 65–82. https://doi.org/10.1075/target.11.1.04ken
Moretti, F. (2000). Conjectures on world literature. New Left Review, 1, 54–68.
https://newleftreview . org / issues / ii1 / articles / franco - moretti - conjectures - on
-world-literature
OED. (n.d.). Loveday. Oxford English dictionary online. https://www.oed.com
Popel, M., Tomkova, M., Tomek, J., Kaiser, Ł., Uszkoreit, J., Bojar, O., &
Žabokrtský, Z. (2020) Transforming machine translation: A deep learning
system reaches news translation quality comparable to human professionals.
Nature Communications, 11. https://doi.org/10.1038/s41467-020-18073-9
Pym, A. (2011). What technology does to translating. Translation and Interpreting
Research, 3(1), 1–9. http://trans-int.org/index.php/transint/article/viewFile/121
/81
Sinclair, S. & Rockwell, G. (2022). Voyant tools (version 2.6.0). http://voyant
-tools.org/
Waugh, E. (1951). Mr Loveday’s little outing. In Work suspended and other
stories: together with Scott-King’s Modern Europe (pp. 7–14). Penguin Books.
Wykes, D. (2016) Evelyn Waugh: A literary life. Springer.
12 (Re)creating Equivalence of Stylistic
Effect
A Corpus-Aided Methodology
Tereza Šplíchalová
12.1 Introduction
We are living in the digital age, and the translation industry has been revo-
lutionised by digitalisation. While there are many subdomains of transla-
tion and interpreting where technology has become inseparably integrated
into the translator’s everyday agenda, the practice of literary translation
appears to be relatively unaffected by this development. The debate on
‘how’ and potentially ‘whether at all’ literary translators can adopt com-
puter-assisted workflows has not passed unnoticed within academia and
studies focused on different facets of this development are increasing in
number (Rothwell & Svoboda, 2019; Kenny & Winters, 2020; Youdale,
2020; Kolb, 2021; Way, Rothwell, & Youdale, 2023), collectively assert-
ing that the productive integration of technology within literary transla-
tion processes is not at all unfeasible. Along the broad spectrum of MT
systems, CAT tools, and methods stemming from corpus linguistics, this
chapter aligns itself with the latter and introduces a corpus-aided method-
ology intended to explore the particularities of translating literary style, or,
more precisely, the intensional structure of a narrative text. This approach
to style stems from a firm belief that narratology and Translation Studies
(TS) would mutually benefit from some further dialogue. Research in TS
that is primarily preoccupied with narratological issues exists (Slater,
2011; Prince, 2014; Hirsch & Lessinger, 2020), but much is still unex-
plored. One such topic would be the potential application of the theory of
fictional worlds within TS. Some have suggested the possibility (Doležel,
1998; Fořt, 2005), and some have already researched it (Kamenická, 2007;
Loponen, 2009), but overall, it does not seem to be well established yet.
Hence, the aim of this chapter is twofold: (i) to introduce intensional func-
tions—an integral part of the theory of fictional worlds—as possible vehi-
cles of style; and (ii) to investigate to what extent these functions can be
described with the aid of a corpus-aided methodology.
DOI: 10.4324/9781003357391-16
220 Tereza Šplíchalová
12.2 Theoretical Framework
To begin with, the relationship between intensional functions and the
broad concept of style, no matter which of the numerous definitions
one chooses to adopt, should be clarified. The premise is that narrative
texts—or fictional worlds—are purely semantic entities that are defined
by their extensional and intensional structures (Doležel, 1998). According
to Doležel, the extensional structure of a literary text encompasses the
semantic elements, both explicitly and implicitly given, that are not tied to
the exact wording of a world-constructing text and can thus be subjected
to paraphrasing. These can manifest as any constituents of the fictional
world in question, be it the entities that inhabit the said fictional world,
its shapes, or its representations. Conversely, the intensional structure
excludes any paraphrasable elements and firmly connects to the very
texture of the text, upon which intensional functions arise. These manifest
as global regularities of texture that affect the structuring of the fictional
world and represent an indivisible element of the style of the original. To
put it differently, the relationship between extensional and intensional
properties of a narrative text is thus not dissimilar to Frege’s notions of
‘reference’ and ‘sense’ (Doležel, 1979), i.e. the relation between a linguistic
expression and an entity to which it refers. As a simplified case in point,
the mere mention of the second planet from the Sun in a narrative text
would introduce the said fictional entity into a fictional world as a part of
the extensional structure. Its particular form, however, be it ‘a morning
star’, ‘an evening star’, or simply ‘Venus’, would then have the capability
to shape the intensional structure by indexing an additional meaning such
as communicating the time of day without explicitly referring to it or
mentioning it at all.
In line with this theoretical framework, my intention is to approach
style as patterns of choices where the ‘patterns’ manifest as intensional
functions. The very straightforward idea this chapter revolves around is
that both structures—extensional and intensional—should be considered
in the process of translation, provided one wishes to recreate a fictional
world as faithfully as possible. I am aware that there are many strategies
one can employ when translating a narrative text, and that there are many
possible goals one can strive to achieve. Therefore, it needs to be made
clear that this chapter operates under the assumption that ‘to translate a
fictional world faithfully’ can be a viable goal of one’s translation activity.
In practice, this would entail producing a stylistically aware translation
that successfully recreates both the extensional and, where possible,
the intensional structure. Such a translation would then prioritise the
faithfulness to the source—the world-constructing text—over any other
motivations. I recognise that translation does not happen in a vacuum and
such an overly specific translation goal might not translate that well into
(Re)creating Stylistic Effect 221
practice, but it could, hopefully, illustrate the relevance of the supplied

theory within TS. Looking at it from a different perspective, an aspiration
to translate a fictional world, with intensional functions included, resonates
with possible translation goals expressed by others. For instance, Youdale
sets out his intention to ‘creatively “reverse engineer” the effects [he]
experience[d] as a reader of the S[ource] T[ext]’ (2020, p. 23), all through
the medium of a computer-assisted methodology based on a combination
of close and distant reading. Youdale expresses his translation goal as
follows:
my overall translation goal was to attempt to achieve equivalence of

stylistic effect: in other words, to recreate in the T[arget] T[ext] the
effects I had experienced as a reader of the ST, an approach which I
believe is shared by many, possibly most, literary translators.
(2020, p. 42)
I share this viewpoint and propose that one of the ways to address the
equivalence of stylistic effect is to look at one’s ST through the lens of
fictional worlds theories; the equivalence of stylistic effect would thus
equate to a successful rendition of a fictional world in another language.
Youdale also emphasises the necessity of uncovering certain ‘“hidden”
layers of information’ (2020, p. 24) that are essential if one wishes to
reverse-engineer the desired effect, which would, in the context of this
chapter, correspond to recovering the fully operative intensional functions.
In TS, this would mean correctly identifying intensional functions in
the ST and making sure they remain operative in the TT as well. While
this chapter only looks at the three functions identified by Doležel—the
naming, saturation, and authentication functions—an unspoken consensus
between researchers interested in fictional worlds theories is that there
are many more intensional functions that are yet to be described. One
of the secondary objectives of employing the corpus-aided methodology
is that it might, in the long run, help identify potential candidate
functions, but it is not a topic that can be discussed within the scope of
this chapter. Instead, the focus is put on the three intensional functions
that are already established. More details on the individual functions
can be found in Doležel’s Heterocosmica (1998), but in a nutshell, the
naming function manifests as a regularity in the use of proper names and
definite descriptions: for instance, a fictional entity can be referred to
by a certain moniker in selected situations only, which can then prompt
the reader to decode the situation accordingly. The saturation function
manifests as a projection of the density of a fictional text onto a fictional
world, which then governs what information is revealed to the reader,
when it is revealed, and in what manner. This can, for example, be an
implicit mention of a culturally specific item that allows the reader to

make an assumption regarding the settings of a novel or certain fictional
location. Last but not least, the authentication function relates to the
narrative modes through which fictional entities mediate fictional facts.
It operates on a degree of truthfulness, endows different narrative voices
with different levels of power to authenticate fictional facts, and allows
the reader to distinguish, for instance, between reliable and unreliable
narration. It may be noted that this chapter has no intention of providing
an exhaustive overview of what these functions can or cannot do; instead,
it details several facets and realisations proper to one novel in particular
and focuses on those aspects that can be illuminated with the aid of
targeted corpus queries.
12.3 Methodology
To reiterate, the aim is to explore the possibility of accessing the
intensional structure by means of a corpus-aided methodology, i.e. with
individual narrative texts conceived as micro-sized corpora consisting of
one novel and its translation. Even though such a limited scope largely
defies the underlying principles of corpus linguistics and the basics of
corpus building, it is not an uncommon measure in TS to base one’s
study on a small-scale corpus consisting of only a few narrative texts (e.g.
Wang & Li, 2011; Ji & Oakes, 2012) or one text and its translation (e.g.
Youdale, 2020). This situates corpus-based studies of literary translation
‘at the intersection of corpus linguistics and digital scholarship’ (Zanettin,
2017), allowing us to align the purpose of one’s investigation with either
linguistic or literary objectives. This chapter pursues the latter, and in
order to do so, it makes use of a powerful tool of choice, Sketch Engine
(SkE), a corpus-management system co-developed under the Natural
Language Processing Centre at Masaryk University (Kilgarriff et al.,
2014). Its flagship function would probably be the word-sketch feature
and accompanying word-sketch grammar that provides an at-a-glance
snapshot of the behaviour of a selected word and its collocational patterns
(Kilgarriff et al., 2014, pp. 3–5), but its appeal is also grounded in the
possibility of readily accessing a multitude of corpora and eventually
creating one’s own. SkE is accessible via institutional or inexpensive
private subscription.
When considering methodology and workflow, my two main concerns
were replicability and user-friendliness. After acquiring an electronic copy
of the ST, The Road (2006) by Cormac McCarthy, and the TT, a transla-
tion into Czech by Jiří Hrubý titled Cesta (2008), both texts were stripped
of any non-essential typographical characters, such as the hyphens used
for word breaks at the end of a line, and loaded into open-source aligner
software1 to create a multilingual file. I then edited and realigned some of

the segments in a spreadsheet and uploaded the edited file to SkE to cre-
ate a parallel corpus while making use of their automatic lemmatisation
and tagging processes. The result is a fully searchable parallel corpus that
facilitates not only aligned browsing but allows for a number of queries to
be run on either of the aligned texts, or both.
Even though these steps appear to be rather straightforward, and the
preliminary searches run through ‘parallel concordance’ yielded some
interesting results, some concerns need to be addressed regarding the
above-mentioned replicability and user-friendliness. The process of rea-
ligning the automatically aligned segments can be time-consuming and
tedious; for literary translators working to deadlines, the benefit might
not be considered worth the effort involved. Having said that, the crea-
tion of a parallel corpus itself does not take longer than a few minutes and
exporting a multilingual file from a CAT tool can be just as instantaneous,
meaning that any literary translator working in such an environment could
still make use of the proposed methodology without having to face any
significant delays.
One more thing to consider is what can be realistically achieved by
comparing a source text and its translation, in the context of this chapter, an
already published one. Youdale summarises that within the entirety of the
literary translation process, methods combining close and distant reading
(i.e. involving some sort of a quantitative method) can be productively
used in four different scenarios, one of them being ‘comparing the ST and
draft translation, with a view to seeing if certain translation goals have
been achieved’ (2020, p. 25). Investigating a translation that has already
been published is indeed past the point of a draft, but it can still yield data
to support the two main objectives of this chapter: to explore the usability
of the proposed methodology, and to illustrate the applicability of the
fictional worlds theory, provided one’s endeavour to translate a fictional
world can be understood as a ‘certain translation goal’.
12.4 Results
The Road tells a story about a father and a son journeying through a post-
apocalyptic wasteland. This fictional world encompasses two domains—
‘before’ and ‘after’ the apocalypse—and different narrative modes are
used to introduce and establish fictional facts falling under the respective
domains. Initial reading indicates that all three intensional functions are
fully operative, in the sense that the regularity in use creates an additional
layer of meaning for the reader to discover, but revealing their distribution
1 https://youalign.com/.
requires some further investigation. Hence, instead of structuring the

following section by the types of analysis one can perform in SkE, the results
are presented in three subsections, each focusing on one of the intensional
functions and the search queries one can use to gain a better insight
regarding their distribution. In his search for computer-assisted analyses
that would be relatively quick and easy to perform and would produce
results that could inform one’s translation decisions, Youdale suggests five
types of potentially usable analyses, namely corpus summaries, wordlists,
keyword lists, n-grams, and measures of lexical richness (2020, p. 50). My
investigation tallies with this selection and concurs, with user-friendliness
and practical application in mind, that simplicity is key and going beyond
the relatively basic analyses would not be productive if one wishes to
recommend them for translation practice. Therefore, the results detailing
the distribution of intensional functions are obtained only with the aid of
SkE functions titled word sketch, wordlist, keywords, concordance, and
parallel concordance.
Before diving into the individual functions, it might be worth mentioning
that SkE makes it possible to form one’s preliminary hypotheses simply
by looking at the information provided under the corpus overview.
For instance, the first thing to come to notice in the case of The Road/
Cesta is the corpus size, comprising 58,685 words in English and 44,999
words in Czech. It needs to be noted that the language pair used in this
chapter is made up of a predominantly synthetic language, Czech, and a
predominantly analytic language, English, so some difference in wordcount
is to be expected as a matter of course. On the other hand, the much
smaller size of the Czech corpus runs counter to the findings reported by
many scholars interested in the explicitation theory, which suggests that
TTs tend to be more explicit than their respective STs and are therefore
often longer. In any case, the difference in corpus size alone could be a
workable starting point for some further investigation. That, however, is
not within the scope of this chapter.
12.4.1 Naming Function
Of the three functions defined by Doležel, the naming function appears to

be the most straightforward one when integrated into TS. The idea is that
different means of singular reference, such as proper names and definite
descriptions, carry different intensions and thus have the capacity to
convey different meanings; the naming function thus arises by displaying
regularity in assigning said references to a fictional entity (Doležel, 1998).
A close reading of The Road quickly reveals that, save for one exception,
the fictional entities are not referred to by any proper names. However,
further investigation in SkE shows that the naming function remains
operative due to regularity in assigning a finite set of definite descriptions.
For the sake of brevity, only the results relating to the two main fictional
entities are commented on here.
Visualising the distribution requires two tasks: to identify the noun
phrases used to refer to the two fictional entities, and to review them in
context. Due to the different tagsets SkE uses for different languages, it is
much easier to complete the first task in the Czech corpus since it is pos-
sible to use the wordlist feature and list only masculine, animate nouns.
The English tagset does not have a gender attribute, so going through the
resulting list of nouns would be slightly more time-consuming, but not at
all impossible. An alternative solution which has proved effective is to use
the concordance feature to search for the third-person singular masculine
pronoun ‘he’, keep only the lines that also include a noun, and see what the
pronoun refers to. Although somewhat inelegant, the search yielded sev-
eral promising candidate terms that were then displayed in context via the
concordance feature for the second task. Some of them were discounted,
and the remainder were examined with the aid of the word sketch feature.
The results show that there are three definite descriptions used for each
entity: the first entity is referred to as the man/father/papa, and the second
entity as the boy/child/son. The collocative patterns brought to light via
word sketch show that the descriptions are not assigned haphazardly,
but alternate, in the case of Entity 1, on the basis of the syntactic frame
and/or collocative meaning (Šplíchalová, 2021). This fictional entity is
referred to as the man in third-person narrative and descriptive passages
only, father when addressed in relation to the boy (thus in possessive
structures only), and papa when directly addressed by the boy. In the
case of Entity 2, the choice of definite description depends on the level
of investment expressed through social/affective meaning (Šplíchalová,
2021). The boy occurs in neutral and descriptive passages, the child in
passages highlighting vulnerability, and (his) son in the most tender and
emotionally charged moments.
After establishing that there is a pattern in assigning the definite descrip-
tions, ST and TT lines were compared to reveal that the pattern is dis-
rupted in the TT; the definite descriptions are treated as interchangeable,
and a fourth description, kluk [the lad], is introduced for Entity 2, as in
the following example:
ST: Then he took the boy with him into the woods.
TT: Pak chlapce [the boy] odvedl do lesa.
ST: He didnt [sic] think the boy could travel much more.
TT: Ten kluk [the lad] už zřejmě daleko nedojde.
(McCarthy, 2006, pp. 42, 57; 2008, pp. 41, 55)
These two segments illustrate that the boy can either be translated as
chlapec [the boy] or kluk [BOYCOLLOQ; the lad]. The naming function has
not been kept intact, which, in combination with treating the rest of the
definite descriptions interchangeably, renders the function inoperative. It
should be noted that the boy is the most frequent noun phrase that occurs
596 times in the ST. In the TT, chlapec occurs 585 times, while kluk occurs
only 31 times so only in a handful of cases. An interpretation that sug-
gests itself would be Toury’s ‘avoidance of repetition’ (1995), i.e. a transla-
tor’s inadvertent tendency to avoid repeating the same noun over and over
again, albeit to the detriment of the otherwise very symmetrical naming
system introduced in the ST. Despite the low number of cases featuring
kluk, introducing the fourth descriptor and treating all of them as inter-
changeable changes the distribution of the naming function and alters the
intensional meaning of the translated fictional world. Alternatively stated,
it robs the readers of the TT of an extra level of meaning that could have
been communicated by introducing a comparable tripartite system. An
attentive translator-reader might be able to spot the pattern just as easily
without any computer-assisted or corpus-aided analyses, meaning SkE is
not needed to reveal the pattern itself but has been instrumental in survey-
ing the overall distribution of the function. I recognise that identifying the
individual definite descriptions can remain a matter of close reading, but
SkE queries, namely those run through word sketch and parallel concord-
ance, proved useful in illuminating their respective behaviour and differ-
ences in TT renditions.
12.4.2 Saturation Function
The second intensional function arises from the notion that fictional worlds
are, by their very nature, always constructed as incomplete; the distribution
of fictional facts into the domains of the determinate, the indeterminate,
and gaps forms the explicit, implicit, or zero texture (Doležel, 1998). In
principle, this function governs the extent to which the fictional world is
‘saturated’ with information and in what manner this is presented to the
reader: either explicitly, implicitly, or not at all. The latter represents an
enticing topic for both narratology and TS and generally refers to pieces
of information the narrative does not provide and leaves up to the reader’s
imagination. Doležel refers to the domain conditioned by zero texture
as ‘gaps’, but it is not uncommon to encounter different terminology
denoting the same concept, such as ‘empty spaces’ or ‘silences of the story’
(Lessinger, 2018).
In terms of relevance to TS, there is more to focus on than just the
narrative gaps. For instance, Kamenická (2007) shows that the saturation
function can be explored against the backdrop of the explicitation
hypotheses to establish a translator’s explicitation profile, but the shifts
between the domains of the determinate and indeterminate can also be
interpreted with the text’s global texture in mind. This would align well
with the findings of Hirsch and Lessinger (2020, p. 756), proposing that in
certain cases, ‘the linguistic explicitation corresponds to an implicitation of
one of the narrative layers of the story, and of the text’s global meaning’.
They illustrate this proposition in situations where making something more
explicit, for instance, a subtle clue, or any part of a text where obscurity
or sense of mystery is valued, intrinsically affects the global meaning of the
text and renders it more implicit. An example would be a series of hints or
clues that the reader is meant to uncover at certain points in a narrative; if
explicitated in the TT, this ‘guessing game’ is not mediated for the readers
of the TL and the resulting structure, with a layer of intensional meaning
missing, can be thus understood as globally more implicit.
With respect to these two approaches, the first logical step would be to
see whether SkE can be instrumental in identifying instances of explicita-
tion. The aligned layout and powerful filtering options of parallel concord-
ance provide an ideal environment for close reading, but a more detailed
query can be devised as well. In line with the findings reported on the use
of names and definite descriptions in The Road, a possible area of inter-
est could reside in the category of changes of logical cohesive ties, which
includes the replacement of personal pronouns with proper names (Blum-
Kulka, 1986, quoted in Hirsch and Lessinger (2020, p. 756)). As there are
virtually no proper names in The Road, I propose that replacing a personal
pronoun with a definite description of choice can, in the context of this
chapter, be considered within the same framework. If so, it is theoreti-
cally possible to put together a simple query in Corpus Query Language
(CQL) and search for pronouns in the ST that are tagged as nouns in the
TT corpus, but I do not want to assume that all literary translators and
researchers are well-versed in CQL building. Having said that, SkE offers
a basic query that allows you to search for a specific example, such as the
pronoun ‘he’ translated as ‘muž’ [the man]. In the case of The Road, the
search yields 113 results of the type in the next example:
ST: He pulled the boy closer.
TT: Muž [the man] k sobě přitáhl chlapce ještě blíž.
(McCarthy, 2006, p. 9; 2008, p. 10)
This might be an interesting option if one wishes to search for lines that
might include an example of explicitation, but I am hesitant to conclude
that occurrences such as these definitively alter an intensional function.
The use of anaphoric/cataphoric referencing in translation does impel the
translator to choose a suitable substitution or drop a definite description,
but the overall impact on the texture seems to be rather trivial. However,
with regard to the global meaning of the text, the same phenomenon can
have much greater impact. There is a short passage in The Road that
describes an encounter with a hostile road rat. A suspenseful skirmish
ensues, culminating in the man shooting the road rat dead. The confronta-
tion begins as follows:
ST: He looked at the road.

TT: Chlap [The manCOLLOQ; the guy] se ohlédl na silnici.
(McCarthy, 2006, p. 38; 2008, p. 37)
Right away, the pronoun is substituted with a definite descriptor chlap

[the guy] to differentiate the newly introduced entity from the man, but
the ensuing skirmish continues to use the same description, the man, for
both fictional entities involved, creating a fleeting sense of suspense in not
knowing which one actually survives:
ST: The man had already dropped to the ground and he swung
with him and leveled the pistol and fired from a two-handed
position balanced on both knees at a distance of six feet. The
man fell back instantly and lay with blood bubbling from the
hole in his forehead.
TT: To už se otec [the father] bleskurychle přikrčil; revolver
držel oběma rukama a vkleče vystřelil z necelých dvou metrů.
Chlap [the guy] okamžitě padl na záda; z díry na čele mu bub-
lala krev.
(McCarthy, 2006, p. 40; 2008, p. 39)
It can be observed in this latest example that instead of two mentions of

the man, each time referring to a different fictional entity, the translator
opts for the more explicit otec [the father] and chlap [the guy]. In the
end, the translator’s decision to interpret the situation more transparently
robs the narrative of some of its intentional obscurities, thus resulting in a
changed, or at least weakened intensional structure. With reference to the
findings of Hirsch and Lessinger (2020), one can also understand this shift
as an example of linguistic explicitation that can, on a more global level of
meaning, be considered narrative implicitation.
To re-emphasise the main point, if translated, the saturation function
seems to be quite susceptible to shifts between the domains of the
determinate and the indeterminate. In general, The Road heavily relies on
the latter and much is never revealed explicitly, such as, for instance, the
nature of the apocalypse or the geographical settings of the novel. There
are, however, some implicit clues pointing to a fictional version of North
America, e.g. a Coca Cola, units of distance, and a sign on a roof-slope
that reads ‘See Rock City’. While it is evident that these clues are perfectly
perceivable with the naked eye, they are also uncoverable with the aid of
SkE; a wordlist of proper names quickly points to the can and the sign, and
a search for numerals immediately shows that their values do not match in
ST and TT. Upon further inspection, it becomes clear that the translator
favoured a domestication strategy and transferred the units of distance
from the imperial system to the metric system. The values were in most
cases re-calculated, as can be seen in the following excerpt:
ST: It looks to be about eight miles or so.
TT: Bude to tak třináct kilometrů. [thirteen kilometres]
(McCarthy, 2006, p. 26; 2008, p. 26)
In connection with the intensional meaning, this decision effectively dilutes

the saturation function to a point where the reader of the TT is presented
with a different set of clues from the reader of the ST. As a consequence,
arriving at the assumption that the narrative is set in a fictional version of
North America might be more complicated.
12.4.3 Authentication Function
Of the three intensional functions, the authentication function appears to

be the least suitable for a distant-reading analysis. It resides in the way
fictional entities mediate fictional facts; broadly speaking, any world-
constructing text has the capacity to utilise different narrative voices and
endow them with different levels of power to authenticate fictional facts
(Doležel, 1998). This can manifest through differences in narrative modes,
or, for instance, through global tendencies perceivable in the way fictional
facts are mediated, be it in relation to form or substance. Form-wise,
McCarthy’s writing is often debated in relation to the idiosyncrasies of
punctuation. In The Road, the punctuation is truly kept to a minimum;
there is not a single quotation mark to signify direct speech, and contracted
verb forms appear without apostrophes. While this might not hinder
comprehension, the absence of some of the frequently used punctuation
marks is prominent enough to wreak havoc in corpus searches.
With an eye on uncovering some general tendencies related to narrative
modes, I decided to make use of the SkE keywords feature, which searches
for both single-word items and multi-word items by comparing data from
one’s focus corpus with data from a selected reference corpus. Naturally,
the results are bound to vary based on the size and contents of the reference
corpus, but in each and every scenario, the top spots in the resulting lists
were occupied by the apostrophe-less contracted verb forms, such as
‘didnt’, ‘couldnt’, or ‘wouldnt’. The purpose of the keywords feature is to
identify words and phrases that are unique to one’s focus corpus, based on
the fact that they appear there more frequently than they would in general
language represented by a reference corpus. In this case, the missing
apostrophe comes across as unique to the ST style and thus as something
that could be considered in translation. After all, it has been shown that
punctuation represents an important dimension of translation, although

often overlooked in favour of meaning (Youdale, 2020, p. 121). The
predicament of the English–Czech language pair, however, is the difference
in use of the affected punctuation mark; in Czech, an apostrophe is the
least common punctuation mark, and its absence is thus not idiosyncratic
in the TT.
However, while running the same search on the TT corpus, an interesting
compensation strategy was detected. Analogously to not marking direct
speech, The Road does not clearly differentiate between the modes of
narration, meaning that passages featuring authoritative narration, internal
monologues, dream sequences, or interactions between fictional entities are
not immediately distinguishable from one another. The keywords results
show that while many of these narrative modes remained obfuscated in the
TT, the passages including direct speech were simultaneously ‘disguised’
by omitting the quotation marks, and ‘unmasked’ by being rendered in
common Czech. The reason this was flagged in SkE is the difference in
form; in the TT, renditions of direct speech integrate inflections taken from
Czech standard vernacular, such as /ej/ instead of /ý/, or non-standard
conditional forms, such as /bysme/. I provide the following example:
ST: It’s the house where I grew up.
TT: Tady jsem jako malej [x malý] bydlel. [This is where I lived
when I was little.]
(McCarthy, 2006, p. 16; 2008, p. 17)
It is worth mentioning that any passages featuring the man’s internal

monologue, a narrative mode leveraging its resemblance to direct speech,
remain in standard Czech. The distribution of the authentication function,
originally operating on a level of obscurity, is thus considerably altered
in the TT. Just like the changes made to the saturation function, these
alterations are not uncoverable without a distant reading approach,
but SkE certainly facilitated confirmation of my initial hypothesis. For
illustrative purposes, after having all the words ending with /ej/ listed,
the aligned segments were checked against the context note to confirm
they all feature direct speech. The context note is accessible through the
parallel concordance feature and displays a passage of selectable length
that details where in the narrative a segment comes from. That being the
case, it made the task of assigning a segment to its respective narrative
mode quite efficient.
12.5 Conclusion
This chapter demonstrates that approaching intensional functions as
vehicles of style creates many opportunities and poses many challenges.
Due to the scope of this chapter, many other issues were not addressed,
and it is thus advisable to view the findings presented as a cautious survey
of the intersection of the disciplines involved rather than a comprehensive
overview. The proposed methodology relies on a close-reading analysis
supported by, not entirely based upon, a distant-reading approach put into
practice via Sketch Engine. The objective was to see whether it is possible
to make use of corpus-aided methods to uncover differences in the ST and
TT intensional functions, and the results are, to an extent, affirmative.
In a general sense, querying the one-novel corpus appears to be the most
advantageous if one works towards a verification of hypotheses based on a
close-reading analysis. This creates a general framework of what to look for
and what software functionalities to use but leaves enough space to discover
the unexpected. The targeted queries in parallel concordance, for example,
quickly reveal inconsistencies in terminology that can be interpreted as
causal agents of changes and alterations of intensional meaning. The rest
of the features mentioned above (i.e. word sketch, wordlist, and keywords)
are indeed instrumental, but highlight the importance of knowing what
to look for beforehand. It does not need to be understood as a downside,
though; this reflection consolidates the usefulness of a method combining
close and distant reading by proposing that any hypotheses formed by
the act of reading can be subsequently confirmed or disproved in Sketch
Engine. The exception would be the naming function, given that an SkE
query can, albeit somewhat inelegantly, reveal the definite descriptions in
use and show the distribution of the naming function without having to
search for the individual elements well in advance. It has been suggested
that the naming function is perhaps the most suitable for this type of
investigation, and the results confirm this. By running a set of searches
in SkE, one can effectively identify the definite descriptions used for
different fictional entities, obtain an overview of their distribution with an
aligned list of their renditions in the TT, and gain a deeper understanding
of their collocative patterns. Regarding the second intensional function,
the saturation function, the SkE queries seem to be usable only with a
prior knowledge of what to investigate. As discussed earlier, the saturation
function can be linked to a so-called translation universal, namely the
explicitation theory, and processed accordingly. Lastly, recovering the
distribution of the authentication function in both ST and TT is a task that
benefits the most from a close-reading analysis, but cross-referencing one’s
findings in SkE can eventually help illuminate the shifts affecting certain
narrative modes. All in all, having an aligned text with the possibility of
zooming in on any interesting shifts or occurrences helped to shed some
light on the differences in ST and TT intensional structure.
My aim has been to present a method that would be, albeit fractionally,
applicable to literary translation practice. With some reservations detailed
above, the measures taken to ensure user-friendliness and replicability,

such as choosing a widely accessible tool or limiting the investigation to
basic queries only, seem adequate enough to still allow the recovery of
intensional meaning without necessarily overwhelming the human sitting in
front of a screen. Implementing the findings in one’s translation draft could
thus result in a better-informed translation, one that takes into account
stylistic nuances that might be intrinsically linked to the underlying texture
of a narrative text. This leads me to believe that in the context of literary
translation, the practical application of both fictional worlds theory and
computer-assisted analysis is worth exploring further.
References
Doležel, L. (1979). Extensional and intensional narrative worlds. Poetics, 8(1–2),
193–211. https://doi.org/10.1016/0304-422x(79)90020-2
Doležel, L. (1998). Heterocosmica: Fiction and possible worlds. Johns Hopkins
University Press.
Fořt, B. (2005). Úvod do sémantiky fikčních světů [An introduction to semantics
of fictional worlds]. Host.
Hirsch, G., & Lessinger, E. (2020). Obscuring the speaker’s stance: When
explicitating results in implicitation. Meta, 65(3), 745–765. https://doi.org/10
.7202/1077412ar
Ji, M., & Oakes, M. P. (2012). A corpus study of early English translations of Cao
Xueqin’s Hongloumeng. Quantitative Methods in Corpus-Based Translation
Studies, 177–208. https://doi.org/10.1075/scl.51.07ji
Kamenická, R. (2007). Dreaming the original: Original and translation as
two different fictional worlds. In K. Vránková & C. E. Koy (Eds.), Dream,
imagination and reality in literature (pp. 15–21). Jihočeská univerzita v Českých
Budějovicích. https://www.pf.jcu.cz/stru/katedry/aj/doc/sbaas01-kamenicka.pdf
translator’s voice. Translation Spaces, 9(1), 123–149. https://doi.org/10.1075
/ts.00024.ken
Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., Rychlý,
P., & Suchomel, V. (2014). The sketch engine: Ten years on. Lexicography,
1(1), 7–36. https://doi.org/10.1007/s40607-014-0009-9
Kolb, W. (2021, May 13). Literary translation and post-editing: Priming effects
and engagement with the text [Conference session]. Computer-Assisted Literary
Translation Conference, Swansea. https://docs.google.com/document/d/e
/2PACX-1vRWAAgOpMCWqGniXJAULzCra1e4We2U7lQj53e7qbKNIgH
vjLGmHrwlsqbqCJCQIwJ11tztjQVmXMu6/pub
Lessinger, E. (2018). Genesis of a poetics of silence. Sillages Critiques, 25. https://
doi.org/10.4000/sillagescritiques.7421
Loponen, M. (2009). Translating irrealia – Creating a semiotic framework for
the translation of fictional cultures. Chinese Semiotic Studies, 2(1), 165–175.
https://doi.org/10.1515/css-2009-0117
McCarthy, C. (2006). The Road. Picador.
McCarthy, C. (2008). Cesta (J. Hrubý, Trans.). Argo (Original work published
2006).
Prince, G. (2014). Narratology and translation. Language and Literature:
International Journal of Stylistics, 23(1), 23–31. https://doi.org/10.1177
/0963947013510647
Rothwell, A., & Svoboda, T. (2019). Tracking translator training in tools and
technologies: Findings of the EMT survey 2017. Journal of Specialised
Translation, 32, 26–60. https://www.jostrans.org/issue32/art_rothwell.php
Slater, C. (2011). Location, location, translation: Mapping voice in translated
storyworlds. Storyworlds: A Journal of Narrative Studies, 3, 93–115. https://
doi.org/10.5250/storyworlds.3.2011.0093
Šplíchalová, T. (2021). On translatability of an intensional function within computer-
assisted literary translation. The ESSE Messenger, 30(2), 99–107. https://
essenglish.org/messenger/wp-content/uploads/sites/2/2022/02/splichalova.pdf
Wang, Q., & Li, D. (2011). Looking for translator’s fingerprints: A corpus-based
study on Chinese translations of Ulysses. Literary and Linguistic Computing,
27(1), 81–93. https://doi.org/10.1093/llc/fqr039
embrace translation technology. Tradumática.
Youdale, R. (2020). Using computers in the translation of literary style. Routledge.
https://doi.org/10.4324/9780429030345
Zanettin, F. (2017). Issues in computer-assisted literary translation studies.
inTRAlinea Special Issue: Corpora and Literary Translation. https://www
.intralinea.org/specials/article/2254
Part 4
Applying Specialised Electronic

Tools to Literary Translation

13 The Experiment
Avraham J. Roos
13.1 Introduction
In 1997, Douglas R. Hofstadter published his book Le Ton Beau de Marot
(Hofstadter, 1997) with multiple English (re)translations of a minor
French poem by the Renaissance poet Clement Marot. These translations
were created by Hofstadter’s friends, students, colleagues, family, and
noted poets and translators (not all Francophone) who had been given
instructions by Hofstadter concerning the literary constraints of the original
French poem and a literal translation into English. His book explores the
meaning, strengths, failures, and beauty of translation.
This chapter describes a similar experiment, conducted at the May
CALT 2019 conference at Swansea University. The aim was to see if
through computer-assisted literary translation tools, participants with no
prior knowledge of Hebrew and minimal understanding of the biblical
and mystical background of a short Hebrew poem, would be able to
collaboratively construct an English rendering of it, in verse, following the
constraints of the original rhyming scheme.
During the workshop, the 30 (mostly) non-Jewish participants were
instructed to ‘translate’ into English an 8-line poem in Biblical Hebrew,
created by the 11th-century French Talmudist R. Yosef Tov Elem Bonfils
(Guggenheimer, 1995). The poem uses a simple AAAA BBBB rhyming
scheme and participants were asked to create a parallel English text using
an identical scheme. Given a corpus of prose translations and digital tools,
participants (individually or collaboratively) not only retranslated these
prose translations into an English-versed version following the original
rhyming scheme, but some also produced a Polish poem and even a
Chinese one (that followed the rules of classical Chinese poetry—each line
with exactly five characters).
This chapter describes the corpus, tools, process, and some of the final
products. It also demonstrates how Google tools were used to create,
record, and later analyse the collaborative attempt. Several of the key
DOI: 10.4324/9781003357391-18
238 Avraham J. Roos
elements discussed in previous chapters in this book, such as corpora use

for translation and keylogging, will be revisited in this chapter.
13.2 Background to the Experiment

This experiment was part of a PhD research project, which compared
106 variant English (re)translations of the Haggadah, a semi-liturgical
Hebrew booklet used by religious Jews on the Eve of Passover. The
Bible commands Jews to ensure that future generations will remember
their wondrous exit from Egypt thus: ‘And thou shalt tell thy son that
day saying: “It is because of what God did for me when I came out of
Egypt”’ (Exod. 13:8). The Passover Haggadah arose to fulfil this precept
of annually relating the story of the Exodus from Egypt, a ceremony ever
since observed by Jews worldwide ‘distinguishing it as humanity's most
ancient and continuous religious communal observance’ (Kahn, 1994,
p. 9). Its Hebrew text is read out by Jews of every religious inclination
on the Eve of Passover, not at the public worship of the synagogue, but
in the intimacy of the family meal. Although the Haggadah text has an
ancient origin and age-old basic principle, the exact text to be said has
undergone a lengthy development with continual changes and additions,
being shaped and ordered over a period of 1800 years, until it reached
its present form. As a result, this multifarious Haggadah has become a
mixture of different genres from straight prose to verse, from narrative
to prayer, from solemn narration to joyous song, and from hymns to
folkloristic roundelays. Consequently, many different linguistic devices
can be identified in the Haggadah. The Haggadah has been ‘translated
into almost every language spoken by Jews throughout their global
dispersion’ (Yerushalmi, 1975, p. 13). The first English-language version
appeared in London in 1770 and since then, hundreds of English (re)
translations have appeared. In this chapter, we zoom in on the difficulty
of translating an 8-line poem located towards the end of the Haggadah
into English.
This poem, ‫‘ חסל סדור פסח‬Hassal Siddur Pesach’ ( = accomplished is the
Order of Passover), added to the mix of texts of the Haggadah in the 11th
century, officially concludes the ceremony (although several hymns and
songs were added after it throughout the ages). The poem’s main message
is that just as the celebrants have done their best to hold the Passover cer-
emonies in perfection this year, they implore God to enable them to cele-
brate the Passover ritual in the rebuilt Temple in Jerusalem the following
year. The length, phraseology, rhyming pattern, and message of this short
poem make it a perfect candidate for an experiment in a 90-minute literary
translation workshop (15 minutes introduction, 30 minutes translation
work, 15 minutes break, 30 minutes discussion).
The Experiment 239
My PhD research into the existent variant translations of this poem

into English revealed that almost all of the English-language transla-
tors who translated it did so in prose, and even the few who used verse
changed the rhyming scheme considerably. This made me wonder if the
conference attendants could do better. Among translation theorists, there
has been much debate surrounding the issue of form vs meaning in poetic
translation (Holmes, 1972; Lefevere, 1975). Some believe that the form of
the poem should take precedence over meaning in translation, indicating
that a poem should be translated as closely to the original form as pos-
sible, even if that means sacrificing some of the meaning. However, the
majority believe that meaning should take precedence over form. They
claim that it is more important to capture the meaning of the poem in
translation, even if that means sacrificing some of the original form. At
the start of the workshop, we discussed this controversy, and most of the
participants agreed that, if possible, it is preferable to attempt to retain
both form and meaning.
13.2.1 The Poem
Table 13.1 presents the Hebrew source text and its transliteration. Hebrew
is dextrosinistral (reads from right-to-left) and simple character compari-
son will reveal, even to non-Hebrew readers, that the last two characters on
the left of the first four lines are the same, as are the last two characters on
the left of each line in the second strophe. This indicates the rhyming pat-
tern AAAA BBBB as can also be discerned from the transliteration on the
left. However, most translators, even those who were professional poets,
opted for a prose translation as is shown below in Table 13.2. Even the
ones that did create versed translations used a different rhyming pattern.
My digitized PhD corpus of over a hundred variant Haggadah translations
dating from 1770 to now, includes only 16 such versed translations. This
raises the question why.
Table 13.1 – Hebrew source text of the poem and transliteration.
Transliteration Hebrew Source Text

Hassal Siddur Pesach KeHilkhato ,‫( חֲסַ ל ִסּדּור ֶפּסַ ח כְ ּהִ לְ כָתֹו‬A)
Kekhol Mishpatto VeHukatto .‫( כְ ּכָל ִמ ְשׁ ָפּטֹו וְ חֻקָתֹו‬A)
Ka'Asher Zachinu Lesadder Otto ‫( ַכּאֲשֶ ׁר זָכִ ינּו לְ סַ דֵ ּר אֹותֹו‬A)
Khen Nizkeh La'Asotto .‫( ֵכּן נִ ְז ֶכּה ַלעֲׂשֹותֹו‬A)
Zakh Shokhen Me'onnah ,‫( זְָך ׁשֹוכֵן ְמעֹונָה‬B)
Komem Kahal Adat Mi Nanah .‫( קֹומֵ ם קְ הַ ל עֲדַ ת ִמי מָנָה‬B)
Bekarov Nahel Nitey Kannah ‫( בְ ּקָ רֹוב נַהֵ ל נִ ְטעֵי ַכ ָנּה‬B)
Peduyim LeZion Bereenah .‫( פְ ּדּויִ ם לְ צִ יֹון בְ ִ ָנּרּה‬B)
Table 13.2 Eleven historical prose translations of this poem by famous Haggadah translators from 1770 until 2012.
Year Name of Haggadah Yudlov Name English Information Publisher

translator
1770 ‫חספ לש הדגה‬ #257/8 A. Alexander First English Haggadah, by Pinted by W. Gilbert London,
printer A. Alexander and UK
assistants
1794 Sevice for the First Two Nights #371 David Levi Published and translated by David Levi, London, UK
of Passover David Levi
240 Avraham J. Roos
1808 Sevice for the First Two Nights #516 Carefully revised and First revision of David Levi’s Hyam Barnett, London, UK
of Passover corrected by Isaac translation by Isaac Levi
Levi
1914 Hagadah for Passover Service #2618 Not listed M. Lehberger & Co.
for the first two nights of Rȍdelheim, Germany
Passover
1928 Hagada. The narrative of #3149 J.D. Eisenstein With illustrations by Lola New York, USA
Israel’s redemption from
Egypt
1934 The HAGGADAH #3482 Cecil Roth Illustrations Donia Nachshen Soncino Press, London, UK
1936 The Haggadah for Children #3593 Rabbi J. P. Rudin Bloch Publishing Company,
New York, USA
1950 Israel Passover Haggadah #4187 A. Greenbaum With commentary by M. American Biblical Encyclopedia
M. Kasher and lengthy Society, New York, USA
introduction by translator
1951 The Haggadah of Passover #4280 Isidore Fishman Edward Goldston & Son,
London, UK
1953 Birnbaum Haggadah #4345 Paltiel/ Philip Montauk Book Mfg. Co, New
Birnbaum York, USA
2012 The New American Haggadah NA Nathan Englander Little, Brown and Company,
USA
The Experiment 241
Rhyme is an intrinsic feature of almost every language. Most cultures

have simple rhymes in nursery songs or chants. Maintaining a poem's
rhyme in translation, however, frequently requires giving it a new inter-
pretation while retaining the essence of the original work, which is a chal-
lenging task because not all languages are equally rhyme-rich. The English
language, for example, though syntactically quite versatile, is rhyme-poor
(Liberman, 2009). This makes translating a versed text from a rhyme-rich
language like Hebrew into English difficult. As a result, prose translations
abound. Table 13.2 presents 11 historical prose translations of this poem
by famous Haggadah translators from 1770 until 2012.1 These digitized
versions were provided to the workshop participants on a dedicated web-
site page as a reference corpus, to ensure that the decisions of the previous
translators were constantly on display to inform the retranslator’s different
options (Rothwell, 2016). For more about corpus linguistics and custom-
built corpora, see Chapter 12 in the present volume.
13.3 The Experiment
After a short introduction about the Haggadah, the poem, and a discus-
sion on the form vs meaning controversy, the 30 workshop participants
from both Anglo-Saxon countries (GB, USA, Canada) and non-Anglo-
phone countries (e.g. Portugal, The Netherlands, Japan, China, Poland),
were challenged to create a versed AAAA BBBB retranslation of this poem.
They were given the choice to either translate the poem into their native
tongue or into English, doing so either individually or collaboratively. A
shared Google Document was provided for collaborative work on which
participants could share their attempts, comment on the draft translation,
and edit it together.
13.3.1 The Corpus and the Digital Tools
The historical English prose translations from Table 13.2 were provided in
digital form as a reference frame for the participants. This was especially
helpful as almost none of the participants could read, let alone understand,
the original Hebrew text. In addition to these 11 English translations from
1770–2012, participants were offered prose translations in 13 other lan-
guages so that each workshop participant had at least one translation in
their native tongue. There were translations in Arabic, Chinese, Dutch
(3), French (3), German (4), Greek, Hungarian (3), Italian (2), Japanese,
1 Referenced at the end of this chapter. These translations were chosen by the author from a
pool of 106 English translations to represent a variety of translation options from a large
time span and from three different countries (UK, USA, and Germany).
242 Avraham J. Roos
Polish (2), Portuguese, Russian (4), and Spanish (4). These translations
were copied from other language Haggadah translations collected by the
author. This corpus of 41 variant translations was uploaded to a desig-
nated Google Sites webpage2 created by the author.
In addition to the corpus, various digital tools were made available to
the workshop participants, some online, others downloadable. The links
to these tools were provided on the aforementioned web page. The online
tools included Thesaurus3 and Reverse Lookup,4 which were aimed at
helping writers find the right word they were looking for. The Reverse
Lookup tool in particular was deemed useful for writers who have the word
on the tip of their tongue but cannot recall it, as typing in a description
would often lead to the word.
To support the workshop assignment of creating a rhyming version of
the translation, a number of rhyming tools were added, including Rhyme
Zone,5 B-Rhymes,6 and Rhymes.7 RhymeZone provides a list of real
English-language rhymes, while B-Rhymes offers near rhymes and Rhymes
allows one to find rhyming words in languages other than English, espe-
cially useful for the multi-lingual workshop participants. An online sylla-
ble counter, HowManySyllables,8 was added to help the participants with
meter and scanning.
In addition to the online tools, links were provided to three download-
able poetry analysis tools: Poemage,9 VersePerfect,10 and RhymeDesign.11
These tools were chosen for their unique features. For example, Poemage
and RhymeDesign provide a visual representation of the complex
sonic devices in a poem, thereby offering a new way to analyse poetry.
Meanwhile, VersePerfect offers a comprehensive toolbox for poets with
a built-in thesaurus, rhyming suggestions as you type, and links to quotes
from famous poems that use the rhyming words being used.
Participants were given 30 minutes to create their translations, armed
with both the corpus and the digital tools. Unfortunately, some of these
tools were not used by the participants, probably due to time constraints,
as will be discussed below.
2 https://sites.google.com/view/calt-2019-swansea-roos/home.
3 www.thesaurus.com.
4 www.onelook.com/thesaurus.
5 www.rhymezone.com.
6 www.b-rhymes.com.
7 www.rhymes.com.
8 www.howmanysyllables.com/syllable_counter.
9 www.sci.utah.edu/~nmccurdy/Poemage.
10 https://verseperfect.en.softonic.com/download.
11 www.sci.utah.edu/~nmccurdy/rhymeDesign.
The Experiment 243
13.4 The Results
Eight individual English translations were created, one of these being a chain
translation via Portuguese. Only two of these actually followed the guide-
lines of an 8-line poem with rhyming scheme AAAA BBBB (more will be said
about these in 13.4.1). Four non-English individual versions were created,
two in Chinese, one in Portuguese, and one in Polish. The only submission in
this group that followed the guidelines was the Polish one (more about these
versions in 13.4.2 below). Seven participants collaborated on creating a joint
version, which will be discussed in 13.4.3 and in 13.5.
During the concluding discussion of the workshop, most participants
expressed pride in their creations but complained that 30 minutes was not
enough to accomplish the mission to their complete satisfaction. They all
stated that the corpus and tools helped them and that they enjoyed the
experience. We will now look at the individual submissions before focus-
ing on the collaborative project.
13.4.1 Insights—Individual English Submissions
The two poems that followed the guidelines were:
I
Passover Seder is here
According to laws and customs we adhere
As we have permitted the uprear
Of the performance that is premier
Oh, Adonai,
Raise up thy
To the place we lie,
Zion, we sing of the nearby.
II
We've done the Seder with all the dishes
Did it like the good Lord wishes
What a blessing to perform it this is
We'll be sure next year not to be remiss-ish
G-d sitting up there in the sky
See your people multiply
Raise us up like flowers growing high
Heading for Israel the saints go by
Both I and II use the word ‘Seder’ which is the Hebrew word for the
Passover ceremony and is used in the original Hebrew text. This kind of
244 Avraham J. Roos
code-switching (foreignization) of translation in Passover Haggadah trans-

lations became a popular trend in the second half of the 20th century as
can be seen from Table 13.2. Their use of it shows that the creators of
these versions were most influenced by the more modern translations in the
corpus. Version I also uses the word Zion for Israel, and the name ‘Adonai’
for God—a typically Jewish name and Hebrew equivalent of ‘My Lord’—
which was a clear indicator that this translator had a Jewish background.
Upon clarification during the closing discussion, it turned out that she,
indeed, was the only Jewish participant in the workshop.
Whereas the creator of Version I, which sounds forced, seems to have
based her poem on rhymes found with the rhyming tool, Version II has a
much more playful flow with rhymes such as ‘this is’, ‘remiss-ish’, not the
kind of rhyme a digital tool would suggest. Version II also much better
expresses the wish that the Divine presence should lead the Jewish people to
the land of Israel, alluding with a touch of humour to the song ‘Let the Saints
Go Marching in’. Both the rhymes and the added humour show how the
human translator is (still) so much better at creating poetry than a computer.
13.4.2 Insights—Individual Non-English Submissions
Besides several non-successful Chinese and Portuguese attempts, the only

version in the group of individual non-English submissions that met all the
guidelines was a Polish one created by Angelika Peljak-Łapińska, author of
Chapter 10 in this book. With her permission we represent her poem here:
III
Porządek uroczystości już spełniony
według przepisów i zwyczajów skończony.
Oby szczęśliwie był powtórzony,
tak jak dziś był wypełniony!
Najczystszy, królujący na wysokości!
Podnieś Swe zgromadzenie z nicości,
zaprowadź Swe plemię ku Syjonu jasności
ku zbawieniu w tryumfie przyszłości!
This Polish poem not only follows the rhyming scheme of the original
(AAAA BBBB), but also stays very close to the meaning of the original
Hebrew text, thus making it a perfect example of both form and meaning
translation. The English translation of the poem reads:
The order of the ceremony has already been fulfilled

finished according to the rules and customs.
The Experiment 245
May it be happily repeated,

just like it was done today!
The purest, reigning in high!
Lift up your gathering from nothingness,
lead your tribe to Zion of light
for salvation in the triumph of the future!
During the closing discussion of the workshop, one of the non-native speak-
ers, Song Runjuan (School of Foreign Studies of Hebei Normal University,
China), expressed dismay over the instructions that the rhyming scheme of
the original had to be used. She claimed that translation is not just trans-
ferring a text from one language to another, but also from one culture to
another. Whereas the AAAA BBBB rhyming scheme might be suitable for
Hebrew and English, Chinese classical poetry has its own rules—each line
needs to have exactly five characters. I asked her if she could create such a
poem as a translation for the Hebrew one. Two weeks later she sent me the
following by email (presented here with her permission):
节庆家宴闭
吾辈遵父言
今年何其幸
来年续此福
高位圣神灵
创世及世人
望您予关怀
世人永欢唱
We present in Table 13.3 below a transliteration, a literal translation, and

an English translation next to the English translation of the original to
show how close the two versions are, notwithstanding the constraining
rules of classical Chinese poetry, proving that with enough willpower,
translation of both form and meaning is possible.
13.4.3 Insights—Collaborative Attempt
Seven participants with mixed backgrounds (four non-native, three native

speakers) worked together to generate a collaborative English version.
They each used their own computer to work on a document opened on the
Google Documents platform. This free cloud-based text processing service
allows multiple users to work simultaneously on the same document and
see each other’s additions, edits, and comments in real-time. All commu-
nication between the collaborators was solely through the platform. The
advantages of using Google Documents are that all collaborators always
246 Avraham J. Roos
Table 13.3 Translation and explanation of the poem’s Chinese translation, sent to

the author by Song Runjuan.
English Hanyu Pinyin Literal Translation English

Translation of Pronunciation Translation of
Hebrew Chinese
Accomplished is jié qìng jiā yàn Festival celebration family The family feast
the order of bì feast over. in celebrating
the Passover. the festival is
over.
According to its wú bèi zūn fù I/we generation obey father Our generation
precept, to all yán words. has followed
its law and its what our
custom. ancestors
have said.
Even as we have jīn nián hé qí This year how lucky (其 qí How lucky we
had the merit xìng is an adverb. No specific are in this
to order it. meaning here. It is used to year!
emphasize the luckiness
and to give a rhythm in
order to fulfil the rhythm
of five characters a line.)
So may we have lái nián xù Next year continue this luck. We hope we
the merit to cĭ fú can continue
fulfil it in the to have this
future. luckiness next
year.
Thou Pure One, gāo wèi shèng High position saint god The God who
who dwellest shén líng spirit (in Chinese,shén is sitting in
on high! 神 and líng 灵 can mean a very high
the same thing. Here I position.
use two, still to fulfil the
rhythm of five characters
a line.).
Redress the chuàng shì jí Create world and world Who has created
congregation chuàng rén human. the world and
that is the human
without beings.
number!
Speedily lead wàng nín yù Hope you give care (您nín We hope you
Thou the guān huái is a very polite word for can give us
offshoots you in Chinese. I don’t care.
of the stock find an equivalent in
Thou hast English. And 关怀guān
planted, huái means care. These
two characters cannot
be separated. They are a
phrase.
Redeemed, to shì rén yŏng World human forever So that human
Zion in song. huān chàng happily sing. beings in the
world will
sing happily
forever.
The Experiment 247
have the latest version to work on, that collaborators can work whenever
and wherever they want, and that the same document is available through
various browsers and operating systems. A disadvantage sometimes sug-
gested is that a team member might accidentally delete something from
the document for all. However, the platform keeps track of all changes
and earlier versions/change histories are easily accessible. This important
feature is further discussed in 13.5.
The collaborative poem thus created was:
IV
A subpar Haggadah?!
A so-la-la Haggadah?!
A blahdiblah Haggadah?!
The Passover rites have been completed
We knew the rules while we were seated
Just as by their order, we’ve not been defeated
May also in future this ceremony be oft repeated.
Oh Lord who is our one true Sire
To be one people we do aspire
Raise us up the uncounted, we are your choir!
Singing ‘Zion is free’, will never us tire
The use of the term ‘rites’ in this version is interesting as none of the his-
torical versions in the corpus used it. It is, however, one of the suggestions
on Thesaurus.com as a replacement for ceremony, which is used in the
corpus participants had access to (see Table 13.2). This version also uses
the word ‘order’ in line three like many late 20th-century translations.
The poem refrains from referring to the request that the Jewish People be
returned to Israel in the last two verses, and this may be a political decision
by the participants. On the other hand, the allusion to the Divine promise
to Abraham that his offspring will be uncountable like the stars in the sky
(Gen. 15:5), is correctly incorporated in this poem, the only poem to do so.
Three playful titles were added to this poem, maybe the participants did
not have time to decide which to choose.
On the whole, this collaborative poem has the best flow and best con-
veys the general message of the original. The joint effort, through the
digital use of Google Documents, seemed to have paid off, and the par-
ticipants expressed great satisfaction with the co-operation, but the real
results for the researcher of technology in literary translations lie in find-
ing out what went on behind the scenes during the process of the crea-
tion, and this can be done using Google document features and add-ons.
248 Avraham J. Roos
13.5 Google Tools for Keystroke Logging

In Chapter 2 of this book, Kolb describes the use of Translog and audio
recordings to document and analyse the decisions and workflow of
translators. The software used to record the keylogging is quite impres-
sive, but I wondered what one could do with free, off-the-shelf pro-
grams. Therefore, the Google Documents Revision History feature and
the Chrome add-on Draftback were used to look at the collaborative
attempt.
The Revision History feature of Google Documents automatically saves
a document after changes are made, so that one can easily roll back to
earlier versions of the same document. Each date-stamped saved version is
viewable, the name of the person who made the last change is documented
(if that person logged in under their own name before editing), and the
changes are highlighted in the document in their colour code. You can
hover over the change to see the name of the person who made it. Scrolling
through these saved versions moves the document back in time. In collabo-
rative work, Revision History thus reveals who contributed most and what
kind of changes each individual team member made. Using this feature
provides valuable data for the researcher to analyse. Draftback takes this
one step further. Called ‘the archaeology of writing’ by its creator, James
Somers (2014), Draftback plays back, in video form, the complete crea-
tion of a document―keystroke by keystroke. Draftback is not a standard
Google document feature but a Chrome add-on for Google Documents,
which can be downloaded from the Chrome Web Store.12 Upon install-
ing, it adds a button in Chrome that plays back every edit ever made in a
Google document, even the ones made before the installation. Watching
a Draftback video is like seeing the document being created in real-time.
With the ‘play in real time’ option, one can even see at what point the
translator(s) paused to ponder and how long this took. Whereas Revision
History provides static snapshots of previous versions, Draftback offers
insights into micro-decisions and revisions. An unfortunate disadvantage
of Draftback, however, is that for privacy reasons it does not show who
made the keystrokes. Still, through some cross-referencing with Revision
History, this can usually be discovered.
Through the use of these two tools, it can be revealed that 986
revisions were made within the allotted 30 minutes. The participants
worked initially in sequence, starting with the first line of the first
strophe, and working their way down. Within ten seconds, the first line
was committed to the page ‘The Passover rites have been committed’,
12 https://chrome.google.com/webstore/detail/draftback/nnajoiemfpldioamchanognpjmocg-
kbg.
The Experiment 249
which set the rhyme for strophe one from the onset. The first strophe
was considered done within 6.5 minutes after which the participants
shifted their attention to the second strophe. Line one of strophe one
never changed throughout the process and made it into the final version,
in contrast to the struggle with the first line of the second strophe. That
line went through several changes getting its final form only 8 minutes
before the deadline:
(7:22) Oh Lord, who rests on high

(9:55) Oh Lord, who rests on higher
(11:06) Oh Lord, who is our desire
(21:30) Oh Lord, who is our one desire
(21:50) Oh Lord, who is our one true Sire
Communication between the translators happened when suggestions and

comments were added for others to see. One of the participants brought
in repeated suggestions to use the rhyming tools, e.g. after having added
the line ‘Oh Lord, who rests on high’ he added in brackets ‘lots of words
rhyme with high: 587 results for words and phrases that rhyme with high’.
Rhyming with ‘high’ caused many false starts, e.g. ‘Raise up your people
who to you cry’. But the problem started when someone suggested for
the final word of the last verse the word ‘Zion’, writing next to it that
this rhymes with high. Most of the time spent on the second strophe went
on trying to make this work, e.g. ‘people who to you cry’ was changed
to ‘people who are cryin’’, in an attempt to make it rhyme with Zion.
Incidentally, this suggestion was made by the same person who created
Version II. The shift to ‘Zion’, ‘cryin’’ led to a revision of the first line to
‘who rests on higher’, none of which seemed to be satisfactory. For almost
a third of the allotted time, Zion stayed in the last line although none of
the rhyming words in the suggested tools suggested any suitable rhymes
to go with it. During this time, the last line underwent many changes with
several side comments added by the participants:
(09:22) Sing a freedom song back to Zion (with apologies to Bob
Marley)
(11:27) Sing freedom all the way to Zion
(11:38) Freedom songs from here to all the way to Zion
(23:23) Freedom songs from here to Zion (with apologies to Bob
Marley and Bony M.) (Wouldn’t get too many points at the
Eurovision)
250 Avraham J. Roos
Eventually Zion was moved to the beginning:
(23:25) ‘Zion is free’, a song we will never tire
That change in line 8 caused a rapid backward domino effect of changes to

line 7 with the rhyming word ‘choir’, then line 6 with an initial typo: ‘To
be one people we asire’, immediately corrected to ‘aspire’ but this inspired
a change in line 5 to ‘Oh Lord, who is our one Sire’. Each of these changes
was made by another member of the group.
At the beginning of the process, all participants worked on the same
lines together, but after about a third of the time, individual translators
started to work, comment, and edit on different lines and different parts of
the poem. At certain times, four participants were typing simultaneously
in three different parts of the poem. The synergy comes through when
suggestions are incorporated and improved upon, a fascinating process
to watch. The humoristic side-comments were kept on screen throughout
the process and deleted only at the very last moment before submission. In
contrast, suggestions for changes or requests for clarification were deleted
the moment the issue was resolved.
These are just a few of the insights into the way translation decisions
were made collaboratively that were unearthed through the use of two
simple tools.
13.6 Discussion
Song Runjuan’s claim that cultural considerations should be taken into
account when translating a poem certainly holds true when translating
a medieval Hebrew poem into classical Chinese. One could argue that it
would make sense for English translators to translate this Hebrew AAAA
BBBB poem into the rhyming pattern of a classical English sonnet (ABAB
CDCD EFEF GG), in which case more lines than the original eight would
be required. Haggadah translator Maurice Myers came closest to this in
his ‘translation in prose and verse’ (Myers, 1920, p. 67), the very first
attempt to create a versed translation of this poem, but as it was not a real
sonnet, one must question his reasons for neglecting the original’s rhyming
scheme:
Our joyful service is complete

In all its order due;
We have observed in fashion meet
Its laws and statutes true.
Lo! as Thou gavest wisdom, Lord,
The Experiment 251
To worship thee aright,

So may we merit the reward
For all we do this night.
From out Thy lofty dwelling-place
In Heaven's heights uprear
Thy lowly people, them whose race
as numberless appear.
Oh plant the chosen stem again
Upon the holy ground,
In Zion let the joyous strain
Of the redeemed resound.
When looking at the work done by the workshop participants, it is self-evident

that the parallel corpus was invaluable for the successful completion of the
task. It provided them with the necessary understanding of the meaning of
the poem to be translated. In addition, it provided translational decisions of
previous translators and offered insight into which phrases in the source text
had generated the most diverse translations and were therefore apparently
more ambiguous. This knowledge enabled them to stay close to the original
meaning in cases when the source was clear and be more creative when the
text allowed it, e.g. historical translations of the second strophe are far more
varied than those for the first. As a result, retranslating the first strophe went
much quicker, with fewer revisions, than the second strophe.
Except for the rhyming tools, the other digital tools provided were
barely used, probably a result of the insufficient time that prevented a
deeper analysis. For some participants, the rhyming tools hindered more
than helped creative thought. Contrary to my assumptions, I did not find
any correlation between language-level proficiency (native vs non-native
speakers) and the reliance on rhyming tools (e.g. both Version I and II were
written by native speakers).
13.6.1 Collaborative Translation
The collaborative work through the Google Documents platform was

extremely successful and productive, proving that digitally enhanced
teamwork can have great benefits. The use of digital tools to analyse the
collaborative work yielded interesting insights and requires further study.
Co-operation in translation is seen by many translation researchers as a
central feature (Fauconnier & Turner, 1998; Alfer, 2017; Zwischenberger,
2020). Theorists have posited that no translator ever works alone. They
claim that it is wrong to think of the ‘individualistic concept of “the trans-
lator’’ because there is always input from other factors and/or individuals
that help shape the translation’ (Risku & Windhager, 2013, p. 42). Famous
252 Avraham J. Roos
historical translation projects such as translations of the Bible were rarely

done by one individual.13 Recently, translation cooperatives have become
popular. These are companies formed by groups of people with different
translational abilities who cooperate on translations. Research has been
done on such collaborative endeavours looking at who called on who’s
expertise, who asked for whose input and what role individuals took (e.g.
Huss, 2018), but surprisingly little study has been undertaken on simulta-
neous collaborative work such as was done in this experiment. The bulk
of research is on the final product more than on the real-time process of
creation.
13.6.2 Group Dynamics
When people are working in a group setting, they often take on distinct
roles and behaviours, as already described by Lewin in the late 1940s. He
stressed the importance of leadership roles, the self-value of team members,
and the need for a supportive community (Lewin, 1947). An interesting
observation gained through Draftback is that there was a clear difference
in the way native and non-native speakers offered their input. The majority
of non-native speakers hedged their suggestions with a question mark or an
explanatory comment. They were less prone to edit the main text directly
in comparison to the native speakers who often edited the text without
hedging. This was either due to lower levels of confidence in their language
proficiency or to cultural differences. It was mostly the Asian participants
who were hesitant to make the changes themselves. Still, the fact that these
non-native speakers participated shows they felt sufficiently confident as
valuable team members, which in turn reinforces the idea that a supportive
community is key for collaborative work. Indeed, in most cases, their
suggestions were accepted by the other team members. Team members
both revised their own contributions (co-operation) or took responsibility
for the text as a whole (collaboration).
It is interesting to note how these digital tools which were designed for
monolingual collaboration can be made to serve collaborative translation.
13.7 Conclusions
To sum up, notwithstanding the fact that none of the Haggadah transla-
tors used the original AAAA BBBB rhyming scheme to create an English
13 Starting with the Septuagint, most Bible translations were created by groups of Bible
scholars due to the scope of the project and in order to avoid bias. A well-known example
is the King James Bible.
The Experiment 253
translation of the poem Hassal Siddur Pesach, participants in this work-

shop showed―through their retranslations―that linguistically it is defi-
nitely possible. They managed to render the poem into a versified English
version notwithstanding the fact that they did not know any Hebrew.
They did so with the use of a corpus with several English and other lan-
guage prose translations. Online rhyming tools helped them adhere to
the original rhyming scheme. The minimal use of the other tools offered
was disappointing, though understandable given the limited time avail-
able. Syllable-count tools could have helped them make a better scanning
retranslation, VersePerfect would have suggested rhymes while sentences
are typed in, providing extra contextual information about highlighted
words, etc. As shown above, the first strophe underwent several revisions
until participants settled on the four rhyming words ‘completed’, ‘seated’,
‘defeated’, and ‘repeated’. VersePerfect would have offered these rhyming
words, nearly in the same order as in the final version, upon typing in the
line ‘The Passover rites have been completed’. I strongly suggest repeating
this (or a similar) experiment with more time for participants to explore
these tools.
Team members who collaborated used the Google Documents platform.
Using Revision History and Draftback their collaborative attempt was
analysed to gain insight into the creation process of the translation.
Because of the relatively short time participants collaborated, not all
features of Draftback could be utilized in full. For example, Draftback
generates reports and graphs that reveal in which places in the text changes
were made and when. When translation work is done over a longer period
of time (weeks, months), such a graph visually exposes which parts of
the text were worked on when, thus revealing the translation workflow
process. Draftback can even provide specific feedback on which participant
worked at what times of the day.
Not enough research has been done on how groups of translators not
bound by geographical proximity could collaborate on the same translation
simultaneously. The tools suggested in this simple experiment could and
should be used in future larger and longer experiments to analyse the
group dynamics, workflow processes, translational decisions, and cultural
influences.
13.8 Epilogue
Since this chapter was first drafted, a new player has entered the scene. In
recent years, the field of artificial intelligence (AI) has undergone tremen-
dous progress, with AI now making inroads into various aspects of human
life, including language processing. Among the many AI applications that
have emerged, at the time of writing, ChatGPT stands out as a state-of-the-
art natural language processing (NLP) chatbot developed by OpenAI (see
254 Avraham J. Roos
Section I.2, Introduction, and Section 8.4, both in this volume, for more
discussion on the capabilities of these tools). Through its use of deep learn-
ing and cutting-edge technologies, ChatGPT can engage people in natural
conversations and generate text that closely mimics human-written text
in a variety of styles and formats, including rudimentary poetry and song
lyrics.
Despite previous attempts to generate poetry using machines, includ-
ing as far back as 1845, creating poetry with a specific metre, such
as rhyme, has remained a challenge. However, the latest update of
OpenAI's GPT-3 feels like a step forward in complexity that comes from
integrating knowledge about a wide variety of subjects and styles into
one model that writes coherent text (and GPT-4 has just come out in
March 2023). In November 2022, OpenAI released a new model in the
GPT series that has rhyming capabilities previous GPT models did not
have. As a follow-up to the experiment in this chapter, ChatGPT was
asked to produce an AAAA BBBB rendering of the above-mentioned
poem. This exposed its limitations in following the AAAA BBBB rhym-
ing scheme.
Below are some of its initial attempts:
Accomplished is the order of the Passover,

A holy rite that we must never sever.
It is our duty to keep it in its order,
We need the merit to fulfill this border.
Thou, Pure One, who dwellest on high,
Redress the congregation without a cry.
Lead the offshoots of the stock You have made,
Redeemed, to Zion in a song to be played.
And another one:
Accomplished is the order divine

For Passover in its precepts and sign
As we have had the merit to order it
May we also have the merit to fulfill it
Thou Pure One, your throne high in the sky
Redress the congregation, vast and wide
Speedily lead the offshoots of the stock
Redeemed to Zion in melodious flock
Although these were interesting, they did not comply with the guidelines
provided. A final attempt was to make ChatGPT a member in a collabora-
tive attempt. The beginning was promising:
The Experiment 255
I want to create a poem together with you

I'd be happy to help you create a poem! What kind of poem would you
like to create?
Our poem will be a semi-liturgical poem, a poetic translation of an ancient
poem
That sounds like an interesting project! What is the ancient poem you have
in mind, and what language is it written in?
The poem is in Hebrew
Great! Do you have a specific poem in mind that you'd like to translate
into a semi-liturgical poem, or would you like to create an original
poem in the style of ancient Hebrew poetry?
No, I have a poem. It is called ‘Hassal Siddur Pesach’
Thank you for sharing the name of the poem. Could you please provide the
text of the poem in Hebrew so that we can begin translating it into a
semi-liturgical poem together?
[Input Hebrew text]
Thank you for providing the Hebrew text. Here is a possible translation of
the poem in a semi-liturgical style:
Let us sanctify the Passover seder

In accordance with its laws and customs,
As we were privileged to arrange it,
So may we be privileged to observe it.
The righteous one dwells in his abode,
The assembly rises, who has been chosen?
May we soon lead the shoots of the vine
Redeemed to Zion with joyous song.
This is just one possible translation and interpretation of the original

Hebrew text. Let me know if you have any suggestions or changes you'd
like to make, or if there are any specific elements of the original poem
you'd like to highlight in the translation.
In the collaborative effort that followed, the human participant
provided guidance and direction on what to change, what to rhyme with,
and what kinds of changes were needed to meet the AAAA BBBB rhyming
requirements. The process took almost an hour to complete. The result
was a successful poetic translation that combined the strengths of human
creativity and ChatGPT's NLP capabilities:
Accomplished is the Passover order this night,

In accordance with its laws and its rite,
As we were privileged to arrange it here just right,
May we be privileged to observe it in future with delight.
256 Avraham J. Roos
Oh Righteous One who dwells on High,

Whose guiding hand we trust and rely,
May your congregation soon reach the land so nigh,
Redeemed to Zion with joyous cry.
The present analysis indicates that the current state-of-the-art in AI

is insufficient to achieve an optimal translation of the provided poem
in AAAA BBBB form on its own. Nonetheless, when used as part of a
collaborative effort, AI technology can make a valuable contribution to
the process. It is noteworthy that in the Swansea workshop too, individual
attempts at translation were largely unsuccessful, whereas collaborative
creation achieved greater success.
It is important to emphasize that despite recent advancements in
AI-generated poetry (see Chapter 8, this volume), significant limitations
remain. However, future development of AI tools for poetic translation is
expected, either through co-operation with human translators or through
further refinement of AI technology.
References
Alfer, A. (2017). Entering the Translab. Translation as collaboration, collaboration
as translation, and the third space of ‘translaboration’. Translation and
Translanguaging in Multilingual Contexts, 3(3), 275–290. https://doi.org/10
.1075/ttmc.3.3.01alf
Fauconnier, G., & Turner, M. (1998). Conceptual integration networks. Cognitive
Science: A Multidisciplinary Journal, 22(2), 133–187. https://doi.org/10.1207/
s15516709cog2202_1
Gigenheimer, H. (1995). The scholar’s Haggadah. Jason Aronson.
Hofstadter, D. R. (1997). Le ton beau de Marot. BasicBook.
Holmes, J. (1972). The cross-temporal factor in verse translation. Translator’s
Journal, 17(2), 102–110. https://www.erudit.org/en/journals/meta/1972-v17
-n2-meta256/003078ar.pdf
Huss, J. T. (2018). Collaborative translation. In K. Washbourne & B. Van
Wyke (Eds.), The Routledge handbook of literary translation (pp. 448–467).
Routledge. https://www.academia.edu/37517705/Collaborative_Translation
Kahn, A.D. (1994). The chosen nation Haggadah. CFR Publications.
Lefevere, A. (1975). Translating poetry: Seven strategies and a blueprint. Van
Gorcum.
Lewin, K. (1947). Frontiers in group dynamics. Human Relations, 1(1), 5–41.
https://doi.org/10.1177/001872674700100103
Liberman, M. (2009). Rhymes. Language Log, 8 Dec. 2009. https://languagelog
.ldc.upenn.edu/nll/?p=1946.
Myers, M. (1920). The passover Hagadah. Geographia.
Risku, H., & Windhager, F. (2013). Extended translation: A sociocognitive research
agenda. Target, 25(1), 33–45. https://doi.org/10.1075/target.25.1.04ris
The Experiment 257
Rothwell, A. (2016). Computer-assisted translation (CAT) tools as an aid to

translation creativity: The case of Zola’s La Joie de vivre. In M. De Meo, E. Di
Martino, & J. Thornborrow (Eds.), Creativity in translation/interpetation and
interpreter/translator training (pp. 113–116). Aracne.
Somers, J. (2014). How I reverse engineered Google documents to play back any
document’s keystrokes. Jsomers.net, 5 Nov. 2014. https://features.jsomers.net/
how-i-reverse-engineered-google-docs/
Yerushalmi, Y. H. (1975). Haggadah and history: A panorama in facsimile of five
centuries of the printed Haggadah from the collections of Harvard University
and the Jewish Theological Seminary of America. Jewish Publication Society of
America.
Yudlov, Y. (1997). Otzar haHaggadot. The Haggadah Thesaurus / Institute for
Hebrew Bibliography. Magnes Press [Hebrew].
Zwischenberger, C. (2020). ‘Translaboration. Exploring collaboration in
translation and translation in collaboration. Target, 32(2), 173–190. John
Benjamins. https://doi.org/10.1075/target.20106.zwi
Primary Sources
Below is a list of all the Haggadot presented in Table 13.2, ordered by date.
The reason for this organization is that many do not include an author,
editor, or name and most use the same title. It was, therefore, deemed more
effective to order them by their most distinctive feature, the publication
date, especially as Table 13.2 presents them this way. The Yudlov numbers
refer to a comprehensive Haggadah bibliography published by the Hebrew
University, Israel (Yudlov, 1997).
14 Augmenting and Informing the
Translation Process through
Workflow-Enabled CALT Tools
Sasha Mile Rudan, Eugenia Kelbert,
Lazar Kovacevic, Matthew Reynolds, and
Sinisha Rudan
14.1 Introduction
CALT is an emerging field in the process of defining itself, but it is already,
we feel, a misnomer. Adding an ‘L’ (Literary) to the pre-existing notion
of Computer-Assisted Translation (CAT) creates continuity with CAT
research; Machine Translation (MT) is another key technological precur-
sor (cf. Hadley et al., 2022). However, by taking this course of action the
field is perhaps doing itself a disservice in overlooking the potential of
the ‘A’ (or assistance) in a literary context where CALT can qualitatively
shape a translator’s work rather than merely allow a translator to perform
a given task faster and more efficiently. In other words, the ‘A’ in CALT
should ideally stand for translator augmentation (see Lommel, 2018), and
we explain this in what follows.
CAT and MT may have changed the experience of many users, but they
still receive mixed reviews from some professional translators (Penkale &
Way, 2013; Way, 2013). The current placement of CALT within the same
paradigm is one reason why CALT tools are perceived as inhibiting the
creativity and freedom of the literary translator (Russo, 2022, pp. 30–31).
Additionally, they are associated with cognitive friction together with the
inability to freely move the translation across various tools of the potential
computer-assisted toolkit, which gives rise to the (largely correct) assump-
tion that literary translators are generally sceptical when it comes to trans-
lation technology (Russo, 2022, p. 26).
This chapter seeks to look beyond the tradition where technical accuracy
has come to define, yet limit, the field’s ambitions. It thus reconsiders the
potential of technology in literary translation as a shaping factor that can
augment translation practice, in close dialogue with advances in translation
theory. Today, most translators use non-CAT-specific technology on a daily
basis (Russo, 2022, p. 28) in ways that anticipate the potential of CALT to
go, with the help of CAT/MT, beyond a focus on efficiency or outsourcing
towards controlled shifts to the translation process. A modern CALT tool,
we argue, must embrace a wider definition of technology that has hitherto
DOI: 10.4324/9781003357391-19
Augmenting and Informing 259
been off-limits to literary translation to: (i) inform translation (search

engines, dictionaries etc.); and (ii) enable, albeit indirectly, processes better
suited for literary translation (such as editing).
In short, our motivation and primary research question revolves around
rethinking the current approach to building a CALT tool and changing
the very relationship between the CALT tool and the translator using it.
Rather than prodding the translator with suggestions, the new generation
CALT toolkit ought to be designed to empower (augment) translation
practice, keep the human on top of the socio-technological wave, and
avoid the technological singularity (Kurzweil, 2005) as an unwanted
scenario. Some, like Van der Meer (2021), have embraced the notion of the
almighty machine and declared human translators to be redundant; Way
(forthcoming, 2024) provides a convincing rebuttal and offers comfort
for translation professionals by declaring that ‘the singularity will never
transpire in the area of translation’.
Our approach to CALT development draws largely on our experience
of working as part of the AHRC-funded Prismatic Jane Eyre project at
Oxford University.1 This project has focused primarily on translation
variation in corpora ranging across versions in over 20 languages which
presented a global novel’s translation trajectories in the context of
re-creation through translation. Supporting this analysis computationally
has shaped our vision of CALT as a potential contribution in the same
context. This eventually formed some of our conclusions on translation as
a process, its outcomes as observed through both close and distant reading
(Moretti, 2013), and the mechanisms for informing and augmenting
translators and their translation processes. In other words, we obtained
observable scenarios of how this ‘post-factum’ analysis of translated work
could be used ‘in-vivo’ in the translation process to shape future CALT-
tool development. Our practical work on tool development in stylometry,
augmented reading (a hybrid of distant and close reading) and creative
writing (with the support of bilingual/multilingual texts) forms the basis
for a CALT infrastructure project. Finally, extensive work on modular,
collaborative augmented platforms driven by externalised workflows
(Rudan et al., 2021b) has significantly shaped the proposed paradigm shift
in CALT tool design put forward in this chapter.
Below we summarise key areas where we contend the most impactful
improvements can be made to existing CALT tools; they motivate our
work on improving such tools through a proposed design paradigm shift
and are addressed in greater detail throughout this chapter. Such a tool
should:
1 https://prismaticjaneeyre.org/.
260 Sasha Mile Rudan et al.
• Eliminate friction: Literary translation can be a laborious and

underpaid task. Tools can break translation workflows and introduce
cognitive overload for translators with constant visual and mental
switching and comparing across different tools or needing to export/
import work (Hvelplund, 2017). Such unnecessary friction when using
CALT tools is likely to deter a translator from using them (Russo,
2022, p. 33).
• Augment the translation process: Augmentation is not necessarily a
key requirement for literary translators, but it is both an unspoken
expectation and a compelling reason to use a CALT tool.
• Guide and improve the translation process: For the literary canon, a
significant corpus of findings produced by literary and translation
scholars can guide translators in their work.
• Democratise CALT: We argue that a truly augmenting and frictionless
CALT tool must become more transparent and user-driven (primarily
in respect of its workflows), open (primarily in respect of its data) and
‘swappable’ (primarily in respect of its components) in order to allow
the translator to adapt their practice to the translator's needs.
• Support the creative process: Lack of creativity is a primary concern
when it comes to using CALT tools. On the one hand, the inbuilt rigidity
of CAT and CALT tools prevents translators from expressing their
ideas efficiently (see Daems, 2022; Ruffo, 2022) and following their
natural workflows. On the other hand, the lack of qualitative insights,
inter-textual references and encoding best practices or creative working
modalities is a missed opportunity to spark their creativity.
This introduction addresses the motivation for our work and establishes
our research questions. Section 14.2 considers the three most important
(for our work) constituents of the act of translation: translation theories,
actors, and workflows. In Section 14.3 we discuss the translator’s augmen-
tation through CALT together with how they are guided through criti-
cal points in the translation process (in canonical settings where previous
translations or literary scholarship help identify important aspects of a
text and the translation process). Having established our motivation, con-
text and informing strategies, and demonstrated the principles through
our CALT toolbelt (named WriTerra), in Section 14.4 we set out the key
aspects of infrastructural change required to accommodate the CALT par-
adigm shift (and address the CALT design challenges) which, together with
the informing strategies, constitutes the main contribution of this chapter.
Section 14.5 presents in greater detail the current implementation of our
CALT tool. It provides a list of the tool’s features and stands as an impor-
tant reference for the components mentioned throughout this chapter.
Finally, we conclude the chapter in Section 14.6.
14.2 CALT in Context
14.2.1 Translation Theories
Ever since the ‘technological turn’ proclaimed by Cronin (2010), transla-

tion, including literary translation, has been seen ‘as an instance of “human-
computer interaction” (HCI)’; this, however, is still ‘not theory-driven but
an ad hoc process’ (Jiménez-Crespo, 2021, p. 314). Nonetheless, if our
goal is to augment rather than just facilitate the human translator, inter-
vention must be theory-driven.
At the time of writing there have been at least seven international
workshops and conferences dealing with topics falling under the CALT
umbrella (see Section I.1 of the Introduction to this book for details),
and collectively these have already started to shift this status quo. Recent
contributions, notably by Youdale (2020), have extended the discussion
explicitly to the ability of CALT tools to inform the translator about the
properties of their own work that a human reader is liable to miss. Such
approaches situate the translated text in relation to the norm in the given
language, other writers or previous translations. Armed with that informa-
tion, the translator may then be able to adjust their work in line with their
translation process and principles.
This important line of inquiry has not yet been taken as far as it ought
to, both in theoretical and in practical terms. Translation theory has
advanced considerably over the past decades, and we would argue that
the translation process has not really caught up. Here lies, in many ways,
CALT’s genuine potential for literary translation. It enables the translator
to see their work through the lens of text analysis tools geared towards
both a particular translation’s style and characteristics associated with
translated texts. For example, the lexical and etymological layers of our
Bukvik2 tool allow the translator to compare the source text’s pattern of
foreign elements with a balanced corpus (one designed to be representative
of texts in a particular language or genre, in our case modern fiction
written in English) and thus tweak their own approach to word choices
and, notably, foreignisation (Berman, 2021 [1985]) (as just one of the
practical examples of the translation process being informed by translation
theory) in the target text.
Figure 14.1 illustrates Bukvik's etymological component in action in a
stylometric context, showing, for 21 English original and self-translated
novels by three bilingual authors (Vladimir Nabokov, Roman Gary and
Samuel Beckett) with English as their L2, that they all use more Latin-derived
2 Bukvik is a beech grove in Serbian; it is also a derivative of the Russian bukva, meaning a
letter, and a rather obvious pun on the English word book.
Figure 14.1 A
n example of augmenting translators using CALT with Bukvik’s etymological layer to permit comparison of the current
text with other texts and help improve lexical selection.
vocabulary than is found in the Brown corpus.3 While such an etymologi-

cal component in CALT cannot reflect all aspects of foreignisation, it can
systematically inform the translator’s choices. Similarly, machine align-
ment combined with part-of-speech (POS) analysis indicates explicitation
(another translation phenomenon observed in translation theory) in terms
of the increase in syntactic complexity. Claudine Borg’s case-study subject,
the literary translator Toni Aquilina, needed a researcher to painstakingly
analyse his working process to learn that his first draft (done with pen and
paper) was characterised by a high level of explicitation and uncommon
vocabulary (Borg, 2022, pp. 76–77). CALT tools could do similar work,
and contribute significantly to translating, for example, intertextual refer-
ences, retranslation or indirect translation (cf. Poncelas et al., 2020).
14.2.2 Translation-Process Stakeholders (Actors)
The translation processes we present as part of our work involve multiple

stakeholders, such as professional translators, translation and literary
scholars, CALT and Digital Humanities researchers and students of
translation. Different tools, procedures and content are therefore required
to support their work. Though it is possible to keep the three perspectives in
CALT—research, practice and teaching—isolated, ideally the field would
benefit from a continuum and fluidity along the different perspectives and
stakeholders.
AI-powered translation tools learn from enormous corpora of trans-
lated texts (see Section I.2 of the Introduction in this volume) and have
the ability to analyse and imitate a large number of translation processes
and translators. The gloomy cloud of technological singularity (Kurzweil,
2005) hangs over the synergy of humans and machines in CALT (Van der
Meer, 2021), though Way (forthcoming, 2024) provides a sunnier per-
spective. To minimise these risks, we aim to connect translation-process
stakeholders in an interoperable network. Therefore, we have identified
and built an actor-network of stakeholders following the principles of
actor-network theory (Latour, 2007), and introduced both human and
non-human stakeholders. Having them identified and organised into an
actor-network (Figure 14.2) helps to better understand and augment the
translation process as applied to CALT tool design, reduce friction and
increase translation efficiency and creativity.
This would augment a translator with the translation heritage, transla-
tion theory, translation corpora and scholarly findings, and aim to give the
human translator and other stakeholders a competitive and creative edge
over their technological surrogates. At the same time, we will end up with
3 http://icame.uib.no/brown/bcm.html.
Figure 14.2 A set of identified translation process stakeholders (actors).
better-informed and more creative translations and translation processes

and greater translator satisfaction. Our interest in CALT is therefore insep-
arable from a more general concern about the multiple scopes of human
augmentation, collective knowledge and collective intelligence as enablers
of the proposed aim.
Primarily, there is the literary ST (source text) itself and its author.
However, the text does not exist in a vacuum but rather is supported by a
corpus of literature that influenced the writer. There is also the entire network
of characters, places, semantic models and entities that are embedded in the
ST either explicitly or implicitly, and finally the author’s corpus and cultural
backgrounds that contextualise the specific text. The translator does not need
to be aware of all of these, but they constitute material on which the scholar
(another potential stakeholder in the translation process) draws in order to
better understand the author, text or the author’s corpus, which eventually
becomes a valuable input when it comes to better-informed translation.
The translator does not operate in an empty space of the ST and their TT
alone. For canonical texts, there are very likely to be previous translations
that are a valuable source for the translator of aspects that should: (i)
not be neglected; (ii) be improved (if a previous translation were affected
by censorship for example, as was Stanevich’s canonical translation of
Jane Eyre into Russian)4; or (iii) be changed (when it comes to cultural
4 Another example of informative censorship, as revealed through a TM, is E. A. Vizetelly’s

Victorian translation of Émile Zola’s La Joie de Vivre (see Rothwell, 2021; Youdale &
Rothwell, 2022).
changes or the reader’s aesthetic perception or cognition). This should

not apply only to translations in the same target language but also across
different languages which share relevant linguistic or cultural patterns,
e.g. choosing the right moment in the translation of Jane Eyre to switch
between the formal ‘you’ and the intimate ‘you’ in any language that has
that distinction. In such a case, a corpus of all the Slavic translations could
provide a rich space of examples for the new translation (see also Chapter
10, this volume).
Finally, we have a set of CALT tools and CALT augmentation agents
that can inform or act on the text, communicate to the translator and
provide additional insights or knowledge relevant to the translation
process.
14.2.3 Translation Workflows
One key to a successful CALT tool involves enhanced integration with the
real-life (natural) translation process of a translator, rather than ‘bending’
the translator to conform to a CALT-enforced translation process. This is
challenging primarily because the real-life workflows of literary translators
are as varied as they are understudied. Existing research focuses, with the
rare exception of Borg’s in-depth case study, on short passages (Borg, 2022,
p. 30). Yet we know from a flood of anecdotal evidence—translators’ own
accounts of the processes they follow—that variation from one translator
to another can be striking (Rossi, 2018). Some start, like Borg’s subject
Aquilina, by extensive research and familiarisation with the source text
while others prefer to read as they translate. Some translators use word
processing throughout while others, like Aquilina, write their first draft
with pen and paper and type it up afterwards. Some systematically
compare their solutions to previous translations of the same work (see e.g.
Youdale & Rothwell, 2022), while others deliberately avoid consulting
them. Some (like Aquilina) mostly stick to a well-established routine from
one translation to another, whereas others go with the flow.
Consider the simple workflow example in Figure 14.3 showing how
CALT fits in with real-world translation processes today, presented by
Schiaffino (2016). This reasonably straightforward process focuses primar-
ily on segmentation and assumes a completely linear workflow between
draft and revision with only one choice to make. On the one hand, even
though the workflow represents a fairly simple translation process, it still
introduces significant friction as datasets must be exported (T3) and even-
tually reimported (T5) for revision elsewhere, while on the other hand, it
is easy to imagine that a more realistic translation process would require
multiple translation drafts and therefore multiple reiterations over friction
tasks in the workflow (T3 and T5).
Figure 14.3 A
n example of a CALT creative-translation workflow, adapted from
the full workflow proposed by Schiaffino (2016).
Compare this to the process described in detail in Borg (2022, p. 83;

emphases added), where a single step in Aquilina’s fine-tuning phase is as
follows: ‘… phase 3 is cyclical and recursive: a comparative self-revision at
the sentence level is followed by a unilingual self-revision at the paragraph
level. This cycle completed, the translator moves to the next ST–TT
segment and a new sequence begins…’. We can see striking differences
between these two possible workflows: Aquilina’s process is cyclical and
involves continuous transition between working at the sentence compared
to paragraph level and is unilingual compared to comparative text
presentation (see Figure 14.4).
It is important to understand that: (i) the second workflow introduces
significant friction when it comes to hypothetical (as Aquilina primarily
uses Word and dictionaries) use of CA(L)T tools due to continuous
changing of preferences and viewing perspectives; and (ii) the first and
second workflows are too different to be easily supported by conventional
CA(L)T tools. What are required instead, we argue and propose, are CALT
tools that are workflow-based and -driven, and as such can accommodate
a range of translation processes through custom-designed workflows
(designed either by translators themselves or, partially by CALT experts,
at least at the first generation of the tools).
14.3 Toward an Informed Translation Process

14.3.1 Augmentation of the Translation Process
One salient example of the potential of an informative approach to sig-

nificantly augment the translation process in line with advances in transla-
tion theory concerns the translation of patterns within a literary text. The
notion of the translation unit (TU) has gone through multiple re-evalua-
tions but is predominantly defined horizontally (Nord, 1997, p. 69) as a
segment of the text on the page taken sequentially, and this comes with
limitations. In practice, literary translators (e.g. Markovich, 1996) have
long known the importance of translating the same term throughout the
text consistently, for example; indeed, CALT tools often facilitate such
Figure 14.4 V
isualisation of Aquilina’s workflow; the top diagram shows the perceived workflow, and the bottom what was actually
observed.
consistency already. In literary works, systematic reference to the text in

its entirety in translating any given segment goes far beyond translating a
repeated term consistently. Lotman (1977, p. 10) describes how the build-
ing of a ‘series’ in a literary text allows literature to generate and store
information in ways that are inaccessible to other kinds of text. The limi-
tations of the ‘horizontal’ or sequential interpretations of the TU are best
captured in Nord’s term ‘vertical translation unit’ (VTU) which refers to
‘functional units that are not rank-bound, with each unit manifested in
various linguistic or non-linguistic elements that can occur at any level
anywhere in the text’ (Nord, 1997, p. 69; cf. Berman, 1985; Zhu, 2004).
Interpretation of a literary text resides in any number of intricate
patterns; breaking these patterns is liable to affect the text’s meaning
significantly (Berman, 1985, p. 76). Kelbert (2022) traces the impact of the
slightest shifts in the translation of individual elements of such a series on
the physical appearance of a novel’s character. Nonetheless, no individual
translator, no matter how sensitive, can account for all the VTUs in a
literary work; the translation process is also primarily linear (indeed, many
translators prefer to forego reading the ST before starting to translate:
Rossi, 2018, p. 51), making VTUs a likely casualty.
This area is one striking example of the potential of a carefully designed
CALT tool to genuinely augment translation practice by integrating
(supporting) advances in literary theory. CALT offers two strategies: (i)
automatically alerting the translator to potentially significant patterns in
the ST and in the emerging TT; and (ii) enabling the translator to manually
encode the patterns they detect into series to which they are then alerted by
the CALT tool when translating one element of such a series.
The first strategy is supported by a set of the toolkit’s components that
consist of discrete lenses that can be combined to identify potential patterns,
including but extending far beyond repetition. Some are more generic
(lexical) and supported by our Bukvik tool which is in the foundation of
the WriTerra CALT tool. In that scenario, the translator, for example, is
alerted to lemmas that are used significantly more or less frequently in the
ST than in the balanced corpus, as well as to how other occurrences of
these lemmas have been translated so far.
Other relevant Bukvik components are designed specifically to
encourage a vertical approach to translation more generally. Notably, the
tool’s Society of Words component encodes the text as a social network
to identify the best-connected lexical nodes, as well as outliers like rare
words that may have particular significance. These lexical nodes, as
well as keywords identified by the translator and their synonyms and
antonyms, are then systematically brought to the translator’s attention.
This component also supports named entity recognition (NER), POS
recognition and Wordnet5 generalisation, making it possible to identify

words that occur often in proximity to a given character or to focus on
semantic groups and themes such as colours, clothing, vehicles, weather
and others, as well as POS. The translator can, for example, consider verbs
or colours that occur in the vicinity of a particular character, making them
attuned to potential patterns.
In contrast, the AnnoTata component supports the second strategy:
manual creation of VTUs. This component has multiple uses but one of its
key functions is to allow translators to easily create networks of significa-
tions as they read the ST, translate or revise it. It enables translators or
their editors and collaborators to annotate texts with multi-annotation:
an advanced concept of annotation which can consist of multiple single
annotations that can be distributed either over a single text or across dif-
ferent texts (variants or even intertextually). Thus, potential VTUs are
constructed across versions and languages and brought up automatically
whenever the translator hits on an element in the series. Figure 14.5 show-
ing a WriTerra screenshot (split into three parts for the sake of readability)
gives a sense of how this might look in practice for an imaginary situation:
Irina Gurova using LitTerra’s AnnoTata component to create VTUs while
retranslating Jane Eyre in 1999 (with her translation in Part (c) and the ST
in Part (a) of the screenshot). Gurova republished Stanevich’s translation
(in Part (b) of the screenshot) in 1990 and would probably own a digitised
version. It is worth noting that the translator can create multi-annotations
(we call them AnnoTatas) across documents, and provide comments on
each sub-annotation (4th ‘ugly’ in Part (a) or at the top/high level (the
bottom text spanning across all three parts of the screenshot), but also
see and manage them even in the text editing/writing mode (‘некрасив’,
unattractive, literally ‘not handsome’ in Part (c)). The screenshot imagi-
nes her revising her translation using a multi-annotation ‘Rochester’s ugli-
ness VTU’ she had constructed while reading the ST and/or translating. As
she revises a sentence where Rochester is called ‘ugly’, the CALT toolkit
prompts her, allowing her to translate with the entire VTU in mind. She
consequently realises that, while each passage was translated accurately
and a range of adjectives is used depending on context, including ‘очень
некрасив’ (highly unattractive), ‘уродливый’ (ugly), ‘безобразный’ (ugly)
and ‘какой урод’ (what a freak of nature), her preference for the strong-
est Russian root for ‘ugly’ (уродливый) made Rochester appear much
more repellent in Russian than both Stanevich’s translation, which uses
‘некрасив’ (unattractive, literally ‘not handsome’) and the original (as
observed in Kelbert, 2022, pp. 11–12). Thus, the toolkit makes awareness
5 https://wordnet.princeton.edu/.
Figure 14.5 (a), (b) and (c) Using the WriTerra CALT toolset with VTUs.
of VTUs systematic and integral to translation practice without imped-

ing creativity by introducing automatic translation suggestions but rather
helps the translator to encode their considerations of the ST and the TT.
Instead, it is likely to foster creativity by making the translator mindful of
patterns they themselves detected and the possibility of further patterns of
varying significance.
14.3.2 Guiding and Improving the Translation Process
Alongside informing the translation process, the CALT tools discussed here
have the capacity to involve additional stakeholders, notably researchers,
to guide the translator. The versatility of the workflow model means that
most of the toolkit’s components can be used for textual analysis (this
is especially the case for Bukvik and Annotata), which can then be fed
back into the CALT toolkit. One prominent example of this approach
is the work on keywords conducted as part of the Prismatic Jane Eyre
project. Several keywords were selected and voted on for closer study by
the project team and their renderings in the published translations of Jane
Eyre into 26 languages were then traced (see Rudan et al., 2021a, pp. 3–4
for the relevant workflow). Once the ST and a number of TTs are cross-
annotated in this way through LitTerra, this knowledge (together with the
scholarly commentaries and essays on relevant topics) can be used in a
number of CALT settings, from retranslations of the novel (the translator
can be alerted to the keywords and the complexities of their translations
discovered by researchers) to translator training on the corpus of the
annotated translations, and finally to training users on the CALT toolkit.
14.4 CALT Paradigm Shift

To support the strategies we presented in Section 14.3 for tackling the
challenges outlined in the introduction, we argue that there is a need to
change the very way CALT tools are designed and implemented. In the
present section, we present each of the key aspects of the infrastructural
changes required.
14.4.1 Modular Tool
While constructing our WriTerra CALT tool and brainstorming on how

to improve its Translation Memory (TM) feature, we found a reference
(Ortega et al., 2016) describing an experiment on an FMR (Fuzzy Match
Repair) algorithm supporting an unlimited number of parallel SBIs (sources
of bilingual information) controlled with a set of ‘principled restrictions’,
that we were not able to test as regular CA(L)T tools do not support such
flexibility (even though most of the modern CA(L)T tools support some
Figure 14.6 Specification for a component part of a successful CALT tool.
sort of FMR). Equally, the authors had similar issues testing their algo-
rithm with real translators (hence their use of a simulation against the pre-
translated DGT-TM corpus).
This is a clear example which calls for democratising CALT tools to
thus enable: (i) creative translators to embrace the latest CALT features;
(ii) researchers and developers interested in a particular aspect of a CALT
tool to test it in a real CALT environment; and (iii) the provision of a freely
accessible version of a CALT tool.
Primarily, a new generation CALT tool should not be monolithic in
its nature and design but rather modular, and therefore extendable with
critical features adapted to a given translation process workflow. Most
modern tools support extensions through plugins, but these are mostly for
specific well-identified border cases, like importing/exporting documents
or supporting special dictionaries. When it comes to core concepts like
how to form a TU, how to integrate TM functionality, how to annotate
the ST with a third-party tool for the purpose of NER or POS-tagging etc.,
most CA(L)T tools fail and make it impossible to provide an alternative
third-party solution. However, interoperability is one of the keys to full
tool openness. Our CALT design strategy is modular-by-design, with each
functional section of our system being a separate component.
14.4.2 Components
CALT tools should be built from a set of well-defined components that

have a detailed specification, with described input, output, behaviour
and (if available) User Interface (UI). In Figure 14.6 we visualize all
the aforementioned aspects of components such as ColaboFlow6 and
6 https://colabo.space/en/flow/.
DataTalks7 as we use them through WriTerra's foundational frameworks.

We describe each of the aspects semantically and ontologically by means
of well-defined JSON schemas and ontologies.
This makes it possible for any solution provider to supply their own
component to replace or extend a given aspect of the system’s functionality.
In addition, each component has its own descriptive name, a set of tags
that place it in an appropriate scope (category) in the workflow, which
categorises it when it comes to different approaches (to a particular
translation task, say), all of which helps translators to understand the
purposes of the individual components, which in turn facilitates their later
work or allows an AI aspect of the system to more efficiently augment their
workflow. Eventually, components will represent the implementation of
each of the tasks in the tool’s workflow.
14.4.3 Tool Democratisation
Friction in tasks T3 and T5 of the workflow in Figure 14.3 primarily

concerns tool democratisation, i.e. unlocking CALT data and the
translator’s work outcomes for use by other tools in a CALT’s toolset or by
third-party components of the core CALT tool. Even if conventional CALT
tools support the export of data like TMs which are well standardised and
covered with the TMX format, such data liberation is present primarily
only at the boundaries of the tool.
Our approach offers the same mechanism not only at the boundary
of the tool but rather at the boundary of each feature represented by
workflow tasks and implemented by corresponding components (as in
Figures 14.3 and 14.4). The boundaries (inputs and outputs) of each task
(component) are well documented by semantic schemas contributing to the
overall ontology of the CALT tool (see Figure 14.6).
14.4.4 Component Generalisation
Providing and maintaining a CALT tool long-term is a significant

challenge. Omitting features that are not fundamental to the CALT
workflow is inevitable, resulting in additional cross-tool friction for a
creative translator.
Our solution relies on components designed to be as generic as pos-
sible which offers multiple benefits. Primarily, it makes development and
maintenance cheaper and ultimately sustainable, especially compared with
similar CALT platforms. Prioritising generic components enables their
7 https://colabo.space/en/datatalks/.
reuse, leading to fewer dedicated components being required and greater

reusability across multiple projects.
Finally, this helps us to rethink and liberate usability, and the features
provided through each given component, toward more universal scenarios
in which the component can be used. The outcomes are significant: (i)
components that can support a wider range of preferences and approaches;
(ii) a more resilient component interface not vulnerable to any slight change
in user experience or expectation; and (iii) opening up the components and
the constituting system to third-party extensions and modifications.
14.4.5 Workflows
The variety of translator workflows (see Section 14.2.3) represents a

richness of the translation world; this might appear to argue against the
one-tool-serves-all principle that encompasses a wide variety of workflows.
Indeed, one of the main concerns of translators regarding CALT tools is
the inability to carry out their own preferred workflows established over a
long period of time.
To elaborate on this, take the example of configuring the ST-TT
alignment type. A CALT tool can change the alignment type to either the
sentence or paragraph level to adapt to different translation tasks in a
workflow, either first draft or revision (T1a/b and T4 tasks, respectively, in
Figure 14.3). However, such tools are not aware of a translation workflow
and of the need to automatically change the alignment type in the context
of different translation tasks (e.g. the T1a/b and T4 tasks). This naturally
contributes to the friction the translator experiences with a CALT tool, as
they manually need to change the alignment type every time they change
their translation task.
Our approach overcomes these challenges by externalising and
visualising the workflow for a translator as a tactile artefact that they can
manage, interact with and use to visually describe their own workflows
(Rudan et al., 2021b). Each workflow task can condition the CALT tool
to automatically adjust to the translator’s anticipated needs; they can
express which translation tasks should be facilitated in their workflow,
the transitions between them and preferences for each task (for example,
request paragraph alignment level for the T4 task and sentence alignment
level for the T1 task in Figure 14.3).
Additionally, this approach helps CALT tool developers and translation
theory scholars to convert tacit translation knowledge and practices into
tangible and operable knowledge that they can use to understand the
translation process, improve it, or correlate with the translation outcome.
The second example of friction associated with CALT tools concerns
situations when the translator cannot perform an activity inside the current
tool and has to cross a boundary between different tools in the CALT
toolset. There are plenty of examples of such a use-case, but we point

to one where, after finishing a first draft, the translator needs to export
their work and import it into Word (see Figure 14.3) in order to work on
revising the draft. As translators know to their cost, such disruptive actions
break the creative translation process, introducing undue cognitive load
and friction while losing a lot of the information associated with the freshly
written translation (e.g. notes, metadata, comments etc.), and confronting
the translator with potentially insurmountable integration challenges. We
partially overcome this scenario: (i) by providing an open architecture
where each component and workflow is extendable or replaceable so that
the required features can be embedded into a single CALT toolset without
losing the working context or data; and (ii) by generalising features and
the components that are then specialised for specific use-cases. In the
previously mentioned use-case, we can transform the editing experience
from TU-based translation with parallel ST and TT texts (T2) into a pure,
‘zen-like’ TT-only editor (T4) without any visual distractions, but with
the very same set of editor components. Both approaches encourage open-
source contributions and offload the need to develop specialised CALT-
only components.
14.4.6 Perspectives
Translation perspectives provide different views of the CALT tool and the
text to be translated; each perspective adapts the CALT tool and makes the
translating experience appropriate to each particular task of the process.
The above example of switching between TU-based translation with parallel
ST and TT texts and a ‘zen-like’ pure TT-only editor demonstrates two
such perspectives. Others might include perspectives like: (i) initial reading
of the ST; (ii) annotation of the ST; (iii) exploring previous translations;
and (iv) exploring scholarly findings on the ST and any relevant corpora.
It is important that the perspectives should not be seen as equivalent
to workflow tasks (CALT features); each perspective usually consists of
(applies to) multiple tasks and/or components sometimes even the same
set of tasks but with different configurations (like different TT alignment
types).
14.5 Toward a New-Generation CALT Tool

The core of our solutions relies on the following infrastructural components:
I. ColaboFlow: An infrastructure responsible for running the rest of

the platform based on declarative workflows. It helps modularise the
platform into components and tasks with clear boundaries of inputs,
outputs and expectations and eventually combining it into meaningful
workflows. It supports visually designing, running and auditing the

workflows.
II. DataTalks: A set of mechanisms for describing data artefacts and
concepts (e.g. ST, TT, annotations, TU, TM…) through data schemas
organised into a coherent translation ontology. The schemas are used
to rapidly and automatically build UIs for data artefacts (showing
them on screen, allowing them to be edited, etc.) without additional
coding, which makes the platform cheaper to develop, and allows the
system to be rapidly changed and adapted to new requirements.
III. KnAllEdge: A triple data store for semantic representation of knowl-
edge and permitting inference of knowledge. It provides mechanisms
and interfaces for the efficient creation of and search for data, and
the power of a generic graph for storing data, metadata and results
through the CALT system. As an example, a translator might want
to use schemas and ontology to support building an advanced con-
ceptual network of parts of the ST (VTUs, repetition of metaphors,
etc.) and relate them to examples of others’ translations, scholarly
findings, theoretical concepts as well as their own translation corpus,
comments and collaborative discussions.
IV. TopiChat: A component that supports dialogue-related services, e.g.
commenting on facts, and communication with collaborators/editors.
V. WhoAmI: A component that supports representing actors (either
stakeholders, collaborators or entities in the corpora, like characters,
places, dates etc.) and building a comprehensive (social) network of
such actors.
On top of the infrastructural components sit high-level components that

constitute the core features of the CALT tool:
I. Bukvik: A visual online tool for stylometric comparative analysis

of literary corpora and their visualisation for research or text inter-
pretation. It adapts to a researcher’s specific needs, e.g. discussion,
reproduction of experiments, versioning of experiments and data and
visualisation of results.
II. LitTerra: A tool for reading, exploring and presenting (compara-
tively) literary texts with their translations and critical texts on them.
LitTerra enriches texts in many ways, both spatially and tempo-
rally, through entities, writing style, character analysis, interactions
between texts and writers, among others.
III. AnnoTata: A set of components on top of the LitTerra platform that
supports interactive annotations, whether created by a machine or by
a human. Annotations can be multi-annotations, consisting of mul-
tiple annotated sections both inside a single text and across multiple
texts. Annotations can be augmented from a dialogue perspective

through the TopiChat platform.
IV. Cvrkut/WriTerra: An ecosystem for writers and translators to help
them to write creatively, organise their creations and augment the
writing/translation process. Apart from providing translation and
(eventually) author memories and references to cultural/national and
world knowledge, it is aimed at augmenting the writing and translat-
ing process beyond the regular literary experience with stylometric
and comparative cross-text and cross-language insights.
Figure 14.7 (split into Parts (a) and (b) for the sake of readability) demon-
strates the interface of the WriTerra CALT toolset in practice. We can see
the ST, multiple TTs together with the text currently being translated (the
2nd pane in Part (a) of the screenshot), and collaborative commenting (the
last overlay pane in Part (b) of the screenshot) presented side by side, with
embedded multi-annotations (‘AnnoTatas’), semantic editor and stylomet-
ric insights chart fed from the underlying Bukvik workflow and ezGlot
service.8 For more details, see Rudan et al. (2020, 2021a).
14.6 Conclusion
In this chapter, we addressed the major challenges that a translator faces
when it comes to integrating CALT tools into their standard translation
processes. We established a clear set of expectations for an improved
CALT tool and argued for a shift in focus away from ‘assisting’ towards
‘augmenting’ the translator, which reshapes the relationship between the
CALT tool and the translation stakeholders. Consequently, we proposed
a paradigm shift in designing and implementing a modular CALT tool
with flexible workflow support and democratised data content across the
whole translation process, rather than only at boundary points. This work
is informed and illustrated by the analysis of translation processes and
translation theories evaluated on projects such as Prismatic Jane Eyre.
We believe that the proposed approach is in itself conducive to research
developments in and beyond CALT. To give just one example, a workflow-
based infrastructure such as the one outlined here can gather and analyse
workflow models that translators opt for (anonymously or otherwise),
and thus provide a retrospective analysis for the study of the translation
process. Since the augmented translation process is affected by the use of
technology, these changes can also be fed back into the development of
further CALT tools. At the same time, CALT can provide a retrospective
8 www.ezglot.com/.
Figure 14.7 ( a) and (b) An example of using the WriTerra CALT toolset in
translation (using annotations, collaboration, machine alignment and
stylometric charts).
workflow for the study of the translation process, doing away with the
current impediments to the study of the process for longer projects.
Even though we have already implemented the majority of the CALT
components introduced here, we still need to tighten the integration
between the set of tools that inform the translator in their translation
process (primarily Bukvik) and the pure CALT-editing tool (WriTerra) in
order to apply it in real-world experiments with translators (initially, in
translator training environments), and open it up for external research and
evaluation. Enabling such external evaluation and learning from real-life
applications of our toolkit would be the primary focus of our future work.
References
Berman, A. (1985). La traduction comme épreuve de l’étranger. Texte, 4, 67–81.
https://doi.org/10.4324/9780429280641
Borg, C. (2022). A literary translation in the making: A process-oriented perspective.
Cronin, M. (2010). The translation crowd. Tradumàtica: traducció i tecnologies
de la informació i la comunicació, 8. https://revistes.uab.cat/tradumatica/article
/view/n8-cronin/pdf_15
technology: The role of awareness and attitude. In J. L. Hadley et al. (Eds.),
doi.org/10.4324/9781003094159
Hadley, J. L., Taivalkoski-Shilov, K., Teixeira, C. S. C., & Toral, A. (Eds.). (2022).
.4324/9781003094159
Hvelplund, K. T. (2017). Translators’ use of digital resources during translation.
HERMES – Journal of Language and Communication in Business, 56. https://
doi.org/10.7146/hjlcb.v0i56.97205
Jiménez-Crespo, M. A. (2021). The ‘technological turn’ in translation studies: Are
we there yet? A transversal cross-disciplinary approach. Translation Spaces,
9(2), 314–41. https://doi.org/10.1075/ts.19012.jim
Kelbert, E. (2022). Appearances: Character description as a network of signification
in Russian translations of Jane Eyre. Target, 34(2), 219–50. https://doi.org/10
.1075/target.20079.kel
Kurzweil, R. (2005). The singularity is near: When humans transcend biology.
Duckworth.
Latour, B. (2007). Reassembling the social: An introduction to actor-network
theory. Oxford University Press.
Lommel, A. (2018). Augmented translation: A new approach to combining human
and machine capabilities. Proceedings of the 13th Conference of the Association
for Machine Translation in the Americas (Volume 2: User Track) (pp. 5–12).
https://aclanthology.org/W18-1905
Lotman, J. (1977). The structure of the artistic text (R. Vroon, Trans.). University
of Michigan.
Markovitch, A. (1996). Zametki francuzskogo perevodchika Dostoevskogo. In G.
M. Fridlender (Ed.), Dostoevsky. Materialy i issledovaniya (Vol. 12, pp. 255–
9). Nauka.
Moretti, F. (2013). Distant reading. Verso.
Nord, C. (1997). Translating as a purposeful activity: Functionalist approaches
explained. St. Jerome. https://doi.org/10.4324/9781315760506
Ortega, J., Sánchez-Martínez, F., & Forcada, M. (2016). Fuzzy-match repair using
black-box machine translation systems: What can be expected? Conferences
of the Association for Machine Translation in the Americas: MT Researchers’
Track (pp. 27–39). https://aclanthology.org/2016.amta-researchers.3
Penkale, S., & Way, A. (2013). Tailor-made quality-controlled translation.
Proceedings of Translating and the Computer 35. TC 2013, London. https://
aclanthology.org/2013.tc-1.13
Poncelas, A., Lohar, P., Hadley, J., & Way, A. (2020). The impact of indirect
machine translation on sentiment classification. Proceedings of the 14th
Conference of the Association for Machine Translation in the Americas
(Volume 1: Research Track) (pp. 78–88). https://aclanthology.org/2020.amta
-research.7
Rossi, C. (2018). Literary translation and disciplinary boundaries: Creative writing
and interdisciplinarity. In K. Washbourne & B. Van Wyke (Eds.), The Routledge
handbook of literary translation (pp. 42–57). Routledge.
Rothwell, A. (2021). Uses of alignment: Retranslating Zola with/against Vizetelly.
La Main de Thôt, 9. https://www.researchgate.net/publication/359617463
Rudan, S. M., Kelbert, E., Kovacevic, L., Rudan, S., & Reynolds, M. (2020).
Twin Talk: Bukvik+LitTerra+Colabo.Space – An example of DH collaboration
across disciplines, languages, and style (Vol. 2717, pp. 15–29). CEUR–WS.
https://ueaeprints.uea.ac.uk/id/eprint/84216/
Rudan, S. M., Rudan, S., Kelbert, E., Sagic, A., Kovacevic, L., & Reynolds, M.
(2021a). Colabo.Space—Participatory platform for evolving research and
publishing workflows. In G. Berget, M. M. Hall, D. Brenn, & S. Kumpulainen
(Eds.), Linking theory and practice of digital libraries (pp. 112–17). Springer.
https://doi.org/10.1007/978-3-030-86324-1_13
Rudan, S. M., Rudan, S., & Møller-Pedersen, B. (2021b). Extending BPM(N)
to support face-to-virtual (F2V) process modelling. Proceedings of the
9th International Conference on Model-Driven Engineering and Software
Development (pp. 350–61). https://doi.org/10.5220/0010349203500361
/9781003094159
Schiaffino, R. (2016). CAT tools and translation style. https://www.aboutranslation
.com/2016/04/cattools-and-translation-style.html
Van de Meer, J. (2021). Translation economics of the 2020s. Multilingual Magazine.
https://multilingual.com/issues/july-august-2021/translation-economics-of-the
-2020s/
Way, A. (2013). Emerging use-cases for machine translation. Proceedings of

Translating and the Computer 35. TC 2013, London. https://aclanthology.org
/2013.tc-1.12
Way, A. (forthcoming, 2024). What does the future hold for translation technologies
in society? In S. Baumgarten and M. Tieber (Eds.), Routledge handbook of
translation technology and society. Routledge.
Youdale, R., & Rothwell, A. (2022). Computer-assisted translation (CAT) tools,
translation memory, and literary translation. In S. Deane-Cox & A. Spiessens
Zhu, C. (2004). Repetition and signification: A study of textual accountability and
perlocutionary effect in literary translation. Target. International Journal of
Translation Studies, 16(2), 227–52. https://doi.org/10.1075/target.16.2.03zhu
Index
acceptability 56 BLEU 9, 11, 30–3, 40, 132, 134

accuracy 28, 40, 56, 60, 65, 175 brainstorming 122
actor-network 146, 260, 263–5, 276 B-Rhymes 242
aesthetics 2, 91, 95, 106, 145, Brown corpus 263
174, 265 Bubblelines 211
agency: literary translator’s 101; Bukvik 14, 18, 261–2, 268, 271,
post-editor’s 65 276–7, 279
alignment 8, 15, 30, 75, 107, 109–10,
112, 122, 192, 199, 222–3, 231, canonical text 175–6, 260, 264
263, 274–5, 278 canonical translation 16, 109, 260
alliteration 99 CCMatrix 129–30
allusion 99, 247 censorship 107, 264
ALPAC commission 94 character name 99, 206
Alpino 30 charcut 59
ambiguity 2, 60, 63, 120, 150, 251 ChatGPT 9, 163, 169–70, 253–6
anaphora 106 Chinese classical literature
annotation 269, 276–8 143–4, 237
antonym 268 Chinese net literature 142–55
artificial intelligence (AI) 154, 158, chrF 9
253, 273 Chuangshi 143
assonance 99 clarification 120
attention distribution 65 close and distant reading (CDR) 204,
attention mechanism 8, 10, 163 206, 221, 223, 231, 259; see also
attribution 71 distant reading
Audacity 57–8 code switching 244
augmented reading 259 cognition 60–6, 265
augmented translation, 3, 14–15, cognitive benefit 112, 122
107–8, 124, 258, 260, 264, 277 cognitive effort 11, 15, 54, 58, 60,
automatic evaluation 7, 9, 11, 40, 42, 118, 120, 122–3, 145, 196, 275
47, 132, 133, 138; metric 10–11, 16, cognitive friction 258, 260
30, 32, 36, 40, 47, 138 cognitive system 107, 119, 121
average reduced frequency (ARF) 193 cognitive turn 173
cognitive uncertainty 63
back-propagation 7 coherence 17, 112, 120, 122, 174,
back-translation 8, 11, 31 183; discourse 164; stylistic 174;
Baidu Translate 150 topical 164
bias 8, 33, 42, 73, 84, 142, 252 cohesion 10, 12–13, 40–4, 46–7,
Bible 3, 191, 238, 252 120, 206
Index 283
Cold War 93–4 Corpus Query Language (CQL) 192–4,

collective intelligence 264 196, 199, 227
collocation 15, 46, 180–3, 190–1, corpus stylistics 73
194–6, 207, 210, 222, 225 cosine distance 30, 130
combinatory poetics 142 creative writing 259
COMET 30–4, 36–40, 47 creativity 2, 10–11, 13, 16–17, 39, 54,
comics 144 56, 101, 112, 122, 124, 159, 164,
Common Crawl 129 170–1, 174, 177, 179–81, 183–4,
Common European Framework of 196, 251, 255, 258, 260, 263, 264,
Reference (CEFR) 137 271, 277
Communication Studies 145 cross entropy 132
compensation 230 cryptography 93
computer-assisted literary translation culture 14, 16, 60, 106–7, 121–2, 127,
(CALT): future tools 100–4, 123–4, 142–3, 145, 153–5, 171, 173–5,
258–79; paradigm shift 271–9 183, 192, 241, 245, 250, 252–3,
computer-assisted source-text analysis 264, 277; popular 29, 143; target
(STA) 4, 17, 203–17, 222–3, 99, 174
232, 261 culture-specific item 99, 146,
computer-assisted translation (CAT) 4, 153–4, 222
15–16, 91–104, 106–24, 132, 150, cyberspace 143–4, 153
203–6, 219, 223, 258, 260
Computer Science 93, 118, 204 Daoism 144, 154
concordance 4, 110, 113, 121, 123, data-driven analysis 28, 215
194, 223–5, 230; KWIC 211; data schema 276
parallel 78, 224, 226–7, 231 decision-making 16, 53–4, 56, 58, 61,
consistency 93, 95, 98, 196, 211, 65, 107, 119, 173, 183, 251, 253
215, 268 decoder 7–8, 163–4, 166
constraint 11, 17, 56, 77, 155, 159, DeepL 12, 15–6, 28, 31, 33–5,
164, 169–71, 237; poetic 169; 37–9, 42–8, 54, 56, 58, 61–4, 85,
rhyme 165–6, 168–9; topical 165–6 74–8, 82, 108, 113–19, 121, 123,
context 162–3, 183, 193, 214 133–5, 138
copyright 114, 126, 129, 138, 147, deforming tendencies 174
152, 193 Déjà Vu X3 16, 107, 109–13, 115,
co-reference resolution 57 120–1
corpus 8, 14, 17–18, 28, 33, 55, 71–3, deletion 57, 59, 133
83, 85, 92, 123, 129–31, 190–3, 205, democratisation 272–3
237–8, 241, 253, 259–60, 264, 271, description 145
275–6; aligned 192, 195; balanced development set 7–8, 29, 31
261, 268; bilingual 205; comparison DGT-TM corpus 272
72–3, 75, 77, 82–3; domain-specific dialect 99
129; focus 229; general language 129; dialogue 77–8, 81, 85, 99, 206, 208,
in-domain 29; micro- 17; monolingual 210, 212
29, 131; multilingual 191; parallel diction 99
15–6, 29–30, 33, 75, 126, 129–30, dictionary 15, 18, 46, 61–2, 92, 94,
137–8, 164, 167, 189–91, 193–4, 116–17, 122–3, 148, 192, 205, 259,
201, 223, 251, 263; reference 76, 229, 266, 272
241; subcorpora 193; tagged 116; Digital Humanities 71, 203–4, 263
training 132; trilingual 189, 191, 196, digital literature 142
199, 201; validation 132 digitised archive 92
corpus analysis 17, 107, 181 direct speech 78, 215, 229–30
corpus linguistics 4, 14–15, 17, 204, disambiguation 57
219, 222, 241 discourse feature 27
284 Index
distant reading 203–4, 229–30; fees 95–8

see also close and distant reading fiction 43, 71, 93–4, 178, 261;
(CDR) hypertext 142; interactive 142
distribution 210, 215–16, 223, fictional worlds theory 17, 219–32
226, 231 figurative language 17, 106
DocBook 136 filtering 130, 193, 227
domestication 174, 229 fixed expressions 179, 189–91,
Draftback 248, 253 199, 201
dramatic script 99 fluency 28, 40, 45, 56, 60, 65,
121, 174
early stopping 7, 31, 132 foreignisation 244, 261, 263
e-book 10; bilingual 4, 126, 128, form 100–1, 164, 169, 171, 175, 184,
135–8 229, 239
ecological validity 74 formatting 98, 100, 113, 154
Edit Distance 133–4 frequency 71, 82, 203, 208,
edit operation 11, 133 215–16; absolute 38, 83, 208, 210;
editing environment 16, 107, 110–11, distribution 72, 194; list 75; relative
113, 116, 123 72, 76, 83, 208, 210–11
editing grid 110, 114, 123 friction 263, 265–6, 273–4
efficiency 95, 99, 102–3, 153, 216 function word 73, 208
electronic literature 142 functionalist school 92
emotion 60, 91, 95, 145, 167 funstory.ai 154
encoder 7, 163–4, 166 fuzzy match 112–3, 123; repair
encoder-decoder architecture 7, 123, 271
164, 167
English–Polish–Belarusian (EPB) general-domain data 29, 31, 47
Parallel Literary Corpus 17, 189, genre 6, 16, 27–9, 47–8, 75, 127,
192–201 142–3, 147, 150, 154, 164, 183,
ensemble 31, 47, 132 238, 261; city/modern novel 143;
epistemic function 78–80 fantasy novel 29, 99, 143; historical
epoch 7 novel 143; military novel 143;
equivalence 116, 175, 183, 196; romance 27, 29; science fiction 29,
of stylistic effect 220–32; 99, 144; suspense 28, 29; Wuxia
phraseological (PE) 196, 199, 201; 143; see also Xianxia
syntactic 13 Georgetown–IBM experiment 94
error 12, 57, 65, 100, 177; category German Original Literature Corpus 83
40; classification 13; correction 85; German Reference Corpus 75
human 99 glossary 94, 116, 148, 150
error-based metric 9 Google Documents 245, 247–8,
ethical and legal issues 6, 145, 155 251, 253
etymology 261–3 Google Translate 9–10, 12–13, 31, 33,
expansion 120 44, 127, 133–5, 138, 150, 177–82
explicitation 55, 99, 108, 224, 226–8, grammar 40, 42–3, 46–7, 60, 116,
231, 263 150, 190, 194, 206; generative 158
extension 17, 116, 220 granularity 30–1, 42
eye-tracking 65 Gravity Tales 146
group dynamics 253
fan translation 4, 6, 16, 144–7,
150, 152 HBLEU 134–5
fansubbing 152 HEd 134–5
feed-forward network 164 hidden layer 7, 159
Index 285
hidden representation 160, 162, 164 Jinjiang 143

hidden state 7, 164 JSON 273
HNIST 134–5
homogenization 55, 65 keylogging 15, 54, 56, 63, 238, 248
HowManySyllables 242 keyness 72–3, 76, 83
HTER 134–5 keystroke 11, 54, 57–60, 65,
human evaluation 10–11, 28, 35, 40, 134–5, 248; ratio (KSR) 134
42, 47, 133, 166 keyword 16, 69–78, 82–5, 208, 216,
human parity 11 224, 229–31, 268, 271; analysis
human translation (HT) 4, 8–12, 17, 73–5, 78; candidate 75–6; list 73,
30, 35–8, 42–3, 47–8, 54–61, 65, 75, 82, 224; negative 72, 77, 82, 85;
70, 94, 126–7, 137–8, 142, 145, positive 72, 76–7
147, 150–3, 178–9, 181, 183, 256 knowledge inference 276
human-computer interaction (HCI) 261 knowledge representation 276
hypotaxis 79 KonText 14, 17, 189, 192–201
hypothesis space 167
language: global 93; learning 16, 128,
ideology 107, 122 138, 189, 201; less-resourced 17;
idiomaticity 45, 60, 151, 171, 174, literary 2, 16, 65, 106, 171; minority
178, 181–3, 196, 199, 207 191, 194–6, 199, 201; national 191;
imagery 14, 65 pivot 191; status 1, 191;
imagination 2, 226 well-resourced 15
implicature 145 language generation 159, 161–3,
implicitation 227 170–1
in-domain data 27–32, 46 language impoverishment 12–13, 65
InLéctor 128 language industry 206
input layer 7, 159 language model 11, 131, 158,
Inputlog 57 160, 166; multilingual 30;
insertion 57, 59, 133 neural 161
inspiration 122, 171, 190, 195, language pair 5, 8, 12–13, 54, 56,
199, 201 103, 107, 129–30, 138, 145; high
intensional function 219–21, 223–4, resource 18, 27–48, 129; low
229–31; authentication 221–2, resource 28, 129
229–31; naming 221, 224–6, 231; large language model (LLM) 9,
saturation 221, 226–9, 231 163–4, 169
intensional meaning 226–7, 229, latent representation 159
231–2 latent semantic model 165
intensional structure 17, 219–20, law of growing standardisation
228, 231 174, 176
intercultural communication 16, 142, laws of translation 12
152, 155 lemmatisation 223
interference 12, 17, 45–6, 55, 206 Levenshtein distance 133
internal monologue 230 lexical density 55, 70, 211
internet search 13, 110, 123, 145 lexical diversity 12, 55
interoperability 272 lexical network 181
interpretation 38, 58, 60, 63–4, 84, lexical node 268
180, 190, 241, 255 lexical richness 12–3, 55, 71, 224
intersubject variation 58, 65 lexical variety 35–6, 47–8, 57,
intertextual reference 99, 260, 263 70, 74
interview data 71 lexicon 40
irony 99, 108, 207–8, 214 liberal translation 169
286 Index
Lilt 113–14 statistical machine translation

lineation 99 (SMT); post–editing (PE)
lingua franca 191, 199 Marian 31, 131, 134–5, 167
Linguee 116 marker word 78–82, 85
linguistic distance 195 market 27, 55, 92, 102, 127, 143–4,
literalism 57–8, 63, 65 146, 153–5
literariness 2–3, 17, 28, 33–5, meaning 4, 34, 40–47, 63, 73, 79,
37–9, 42–4, 46–8, 152, 180; of 94–5, 97, 99, 118, 158–61, 163–6,
translations 28; prediction 28, 175, 181–4, 191, 196, 208, 212,
33–36, 42, 47–8 220, 223–32, 237, 239, 241, 244–6,
literary domain 14, 27, 53, 251, 261, 268, 275
55, 106 mechanisation 93
literary fiction 16, 27–8, 42 MetaAI 129
literature: interactive 143; intermedial metadata 275–6
143; popular 145 metaphor 2, 6, 17, 41, 95, 99, 120–2,
LitTerra 18, 269, 271, 276 145, 153, 174–6, 276; conceptual
locale convention 40 173, 176; conventional 174–5, 177,
loyalty 60 180, 182–3; creative 173, 176–7,
180, 183; culture-specific 176; dead
machine learning 28, 33, 71, 107, 175; equivalent 173; explication 176;
158–9 foregrounded 176, 183; grammatical
machine translation (MT) 2, 4–5, 176, 183; image 175, 182; linguistic
7, 9, 11, 13, 15–17, 27–8, 30–2, 173; mapping conditions 174;
36, 39–40, 42–3, 48, 53, 55–7, original 174–5, 177–8; patterns 174,
60–5, 69–71, 74–5, 77, 80, 83, 183; retention 178–9; rhetorical
91–104, 106–24, 126–8, 132–6, device 174; sleeping 176; stock
138, 142,144–5, 148, 150, 155, 175; substitution 174, 179, 183;
158, 163–4, 166, 170, 177, 183, translatability 173–4; translation
204–5, 219, 258; adaptive 114; 173–84
author-tailored 16, 48, 126–38; Metaphor Studies 17, 173, 177
constrained 169–71; corpus-based metre 99–100, 242, 254
8; document-level 27–8, 31, 40–41, mistranslation 41, 44, 45
47, 57; domain-adapted 15, 28; ModernMT 132
engine customisation 69; evaluation monolingual data 8, 31
65; general purpose 28, 31, 35; gloss monolingual reader as editor and
translation 101; hallucination 117; translator 150
‘human-in-the-loop’ 14; interactive mora 100
113–16, 118, 123; literary-adapted morpheme 8
15, 27–54, 93; low-resource Moses 30, 132
domain-adapted 15; markers 13; of Mouse-action ratio (MAR) 134
Chinese fantasy novels 142–55; of movable type 3
literature 9–13; priming effect 53, MTUOC-corpus script 130
55, 58, 60–1, 63–5, 85; principles MTUOC-eval 132
6–9; quality estimation 30; raw multilingual model 47–8
13, 54, 59–60, 65, 75, 133–4, 145; multilingual sentence embedding
rule-based 158; sentence-based 28, space 130
40, 47, 95; system building 8, 30;
training 7–8, 14–15, 27, 29–30, 56, named entity recognition
69, 114, 126, 131, 133, 138, 164; (NER) 268
tuning 8, 31–2, 57, 127; see also narrative implicitation 228
neural machine translation (NMT); narrative modes 222–3, 229–31
Index 287
narratology 206, 219, 226 personalisation: author-specific 69;

national literature 92 translator-specific 69–70
natural language processing (NLP) phraseme 189–90, 195
9, 15, 32, 158–60, 163, 170, phraseology 17, 189, 191,
253, 255 194, 238
neural machine translation (NMT) plot 154, 206–8, 215
6–13, 15–17, 27, 30–1, 54, 56, Poemage 242
77, 102, 107–8, 112, 114, 118, poetry 94–5, 99, 167, 254
126–32, 137–8, 163, 166, 170–1, poetry generation 17, 158–9,
177; see also neural network (NN), 164–7
recurrent neural network (RNN); polysemy 57, 78, 116, 181
transformer NMT PosEdiOn 134–5
neural network (NN) 7–8, 10, 13, 17, post-editese 12, 55, 63, 71
30, 107, 158–65 post-editing (PE) 4–5, 10–13, 15–6,
n-gram 9, 37–40, 71–2, 224; overlap 53–66, 70–1, 74, 83–5, 96, 101–2,
37–9 113, 123, 127, 134–5, 145, 147–50,
NIST 133–4 151, 154; effort 16, 54, 59, 134;
NMTWizard 132 guided 116; literary 69
non-negative matrix factorization post-processing 8, 30
(NMF) 165 precision-based metric 9
normalisation 12, 17, 55, 58, 63, 65, preferential change 85
180, 182–3 pre-processing 8, 30, 130–2
normalized time 134–5 pretranslation 123
NoSketch Engine (NoSkE) 192 principal component analysis 71
novel 15, 27, 42, 95, 99, 142, 145, probability distribution 162,
217, 222, 231 164–5, 169
novelty 37, 56 problem-solving 61
productivity 11, 13, 54–5, 85, 98, 106,
OmegaT 132, 150 123, 127, 145
omission 15, 44, 57, 64–5, 180 Project Gutenberg 109
open source 275 project management 92, 103
OpenNMT 132 pronoun 32, 55, 78, 227
Opus-MT 32–5, 39, 47 prose 10, 54, 71, 99, 108, 192,
out of vocabulary (OOV) 7, 46 237–41, 250, 253
output layer 7, 159 proximity 194
overfitting 47 public domain 126, 128, 130, 133,
137–8
paid-reading service 153 publishing 27–8, 57, 66, 127, 205
Papago 150 pull model 147
Paracrawl 29, 129 pun 95, 99
Paragraph Vector Model 34 PunCAT 99, 122
parallel data 8, 10, 31, 129–31 punctuation 30, 44, 55, 99, 179,
paraphrase 175–6, 220 229, 230
parataxis 57 push model 147
part-of-speech (POS) 12, 263
pattern 2, 12, 14–15, 17, 54, 65, Qidian 143, 153
100–1, 150, 154, 184, 215–17, 220, quality 4, 6, 9–11, 13, 28, 30–1, 55–6,
225, 231, 261, 265–6, 268–9, 271 65, 96, 101–2, 126–7, 129–30,
pause 11, 54, 57–8 132–3, 137–8, 145, 148, 150–5,
'peep-hole’ phenomenon 98 163–4, 166, 171, 177, 184, 190,
perlocution 91, 95, 98 192, 201, 204
288 Index
ranking 42, 44, 48, 72–3, 83, Sketch Engine 203, 222–31
101, 122 skopos 183
rationalisation 120 social network (text as) 268
readability 56 software 3, 12, 73–4, 92, 96, 99, 103,
reader: as editor 148; as translator 114, 192, 203, 223, 231, 248
146; survey 33 speech 71, 78–9, 99, 208, 215,
reading experience 127–8, 136, 138, 229–30
148, 183 statistical machine translation (SMT)
recurrent neural network (RNN) 7–8, 6–13, 54, 102, 163, 177
10, 13, 161–3 statistical significance 73, 83
reference 27, 224 stoplist 208, 212–13
register 45, 57, 71, 79, 81–2, 99, 120, string 7–8, 12, 110, 123
145, 212 style 2–3, 14, 16–17, 40–3, 45, 47,
repetition 106, 123, 211, 215–16, 226 60, 65, 72, 95, 99, 106, 108, 120,
retranslation 16, 106–24, 127, 148, 122, 151, 174–5, 177, 179, 183–4,
238, 241, 251, 263, 271 206–8, 215–16, 219–20, 232, 254,
Reverse Lookup 242 276; author’s 69, 73, 127, 180;
revision 4, 15, 56, 112–13, 240, 249, character’s 73; translator's 4, 13, 16,
265, 274 56, 69–85, 261
reweighting 165 stylometry 259, 261, 276, 277
rhyme 17–18, 95, 99–101, 164, 166–8, subconscious correction 64
170, 180, 241, 244, 249, 254–5; subsegment recall 123
constraint 167; prior 166; scheme subsidy 92, 102
168–9, 237–9, 243–5, 250, 252–4 subword unit 8, 30
RhymeDesign 242 support vector machine 71
RhymeZone 242 syllable 100, 166, 242, 253
rhythm 46, 63, 99, 122, 246 synonym 79, 81–2, 118, 121, 123,
180, 182, 268
sandbox 118, 121 syntactic structure 12
screen clutter 118 synthetic data 8, 11, 29, 31, 64
segment 16, 61, 95–8, 101, 109–10, Systran 147
112–14, 116, 118–19, 121–3,
128, 130–1, 134–6, 192–3, 205, tagging 223, 225, 272
211, 215, 230, 266, 268; length target audience 109
120; parallel 126, 129–30, 138, TAUS Dynamic Quality Framework
230; source 130, 136; target (DQF) 40
113, 136 technical effort 54, 58–60
segmentation 7, 30, 95, 98, 103, technological singularity 259, 263
109, 265 technological turn 261
self-translation 261 temporal effort 11, 54, 58–60
semantic model 264 tense 46–7, 57
semantic proximity 118 TER 9, 134
sentence: length 11, 14, 55, 70, termbase 92–3, 96, 98–9, 154
99–100, 207; pair 8, 27, 29, terminology 55, 58, 61–2, 65, 93,
30; structure 79, 113, 148, 95–6, 98, 106, 150, 154, 173,
178, 182 226, 231
SentencePiece 30, 132 test set 8, 29, 33–5, 37, 42, 133
sentiment analysis 163 testing 13, 57
shift 56, 59, 63, 175, 226, 228, 231, Tetrapla 75
249, 268 text: creative 2, 126; cultural 5;
simile 99, 175, 177–9 electronic 3, 109; focus 72;
simplification 12, 55 formulaic 93, 96, 102; functional
Index 289
91; informative 142; literary 2, 277; universal 12, 231; variation

5, 13, 17, 27, 54–5, 69, 91, 93, 259; workflow 95–6, 103, 260,
99–100, 103, 113, 126–7, 142, 265–6
145, 153, 174, 177, 180, 183, 276; Translation Edit Rate (TER) 133
non-literary 2, 55; pragmatic 110; translation memory (TM) 4, 15,
technical 2, 93–6, 102, 142; texture 92–3, 98, 101, 107–10, 112–16,
220, 226–7, 232; types 2, 12, 94, 96 121, 123–4, 150, 264, 271–2, 276;
theme 14, 120, 167, 179, 181, 207, historical 108, 121, 123; match
215, 216–17, 269 109–10, 113, 123; see also fuzzy
thesaurus 15, 116–8, 121, 123, match
242, 247 Translation Studies (TS) 3–4, 6, 13,17,
think-aloud protocol (TAP) 54–7, 70, 91, 93, 145, 173–4, 189–90,
60–1, 65 204, 219, 221–2, 224, 226; corpus-
threat 6, 96, 103, 106, 151 based 6, 70, 72, 222; Descriptive 6
token 35, 74–5, 129, 131, 134, 191 translationese 11, 12, 55
tone 106, 108–9, 207 translator: literary 1, 3–6, 14–16, 28,
tool democratisation 273 54, 56–7, 60, 65, 69, 71, 93, 96–7,
tracked changes 110, 112 99–103, 106–7, 114, 121, 153–5,
Trados Studio 132, 203 176, 203, 216, 219, 221, 223, 227,
training data 7–8, 10, 13, 29–30, 33, 258, 260, 263, 265–6; satisfaction
37, 39–40, 46–7, 114, 132, 158–9, 97, 264; technical 92, 96, 99, 103;
161, 183; monolingual in-domain training 6, 271, 279; working
32; synthetic 11 environment 58, 74; working style
training set 8 59–60, 65
transformer NMT 8–9, 13, 30, 32, Translog 57, 58, 248
132, 163–4, 167 type 35, 72, 74
translation: as a process 259, 261, type-token ratio (TTR) 35, 70, 74
263, 274, 277; candidate 59, 122,
167; co-translation 6; collaborative undertone 208, 216
6, 245–54, 259; commercial 2, update rate 153
4, 14–15, 47, 91–2, 107, 110, update volume 154
113, 121, 138, 153, 155, 190; user-activity data 74
communicative 176; community user-generated translation 152
152; crowdsourcing 6, 152;
evaluation 6; from scratch 11–12, validation steps 132
16, 53, 61, 65, 70, 74; fully Vecalign 30
automated high-quality literary vector 7, 30, 130, 160, 164; paragraph
4; gold-standard 8, 30; historical 34; representation 130
107–8, 122, 240–1, 247; heritage verse 18, 164–6, 237–9, 249
263; indirect 263; literal 64, 166, VersePerfect 242, 253
168, 196, 237, 245; literary 91–2, versioning 4, 276
101, 103; literary vs. non-literary vertical translation unit (VTU)
1–4; model 129, 167; novel 10, 127; 268–71, 276
ontology 276; perspectives 275; video game 144
phraseology 189–201; platform visualisation 4, 14–15, 17, 203–4, 208,
144–5, 147, 154; poetry 9–10, 17, 211, 216–17, 267, 274, 276
142, 159, 166–70, 256; quality voice 106, 122, 215; authorial 108–9;
estimation 30, 32; reference 7, 133; narrative 108, 222, 229;
semantic 175–6; strategy 101, 109, post-editor’s 55, 65; translator’s 16,
208, 215; technical 92, 96–8, 101–3; 56, 65
technology 13, 94, 101–3, 107, 122, voice recording 15, 57–8
204–6, 217, 258; theory 260–3, Voyant Tools 17, 203–17
290 Index
web novels 145–6, 153 WordSmith Tools 73–5

Webnovel 147 workflow 93, 96, 98, 102–3, 203, 206,
weight 7, 30, 132, 160, 164–5 216, 222, 248, 253, 260, 265–7,
Wiktionary 165 271–5, 277, 279
word cloud 208, 210 world knowledge 16, 277
word co-occurrence 34 WriTerra 260, 269–71, 273,
word embedding 159–63; language- 277–9
agnostic 160; monolingual 160; writing conventions 127
multilingual 160 writing technology 3
Word Error Rate (WER) 133 Wuxiaworld 146–7, 150, 152
word processing 13–14, 92, 145, 265
word representation 161 Xianxia 16, 142–155
word sketch 222, 224–6, 231
wordlist 194, 224–5, 228, 231 Yandex 150
Wordnet 269
wordplay 2, 6, 179–80 Zongheng 143

Computer-Assisted Literary Translation (Routledge Advances in Translation and Interpreting Studies Series)

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Computer-Assisted Literary Translation (Routledge Advances in Translation and Interpreting Studies Series)

Uploaded by

Copyright:

Available Formats

Computer-Assisted Literary Translation

This collection surveys the state of the art of computer-assisted literary

Andrew Rothwell is Professor Emeritus of French and Translation Studies

Andy Way is Professor in the School of Computing and Deputy Director

Roy Youdale is Research Associate in Translation Studies at the University

Computer-Assisted Literary Translation

For more information about this series, please visit: www​.routledge​.com​

Edited by Andrew Rothwell, Andy Way

1 Literary-Adapted Machine Translation in a Well-Resourced

2 ‘I Am a Bit Surprised’: Literary Translation and Post-Editing

3 Mark My Keywords: A Translator-Specific Exploration of

4 MT and CAT: Challenges, Irrelevancies or Opportunities for

5 Retranslating Proust Using CAT, MT and Other Tools 106

6 Author-Tailored Neural Machine Translation Systems for

7 Machine Translation of Chinese Fantasy (Xianxia) Novels:

8 Up and About, or Betwixt and Between?: The Poetry of a

9 Metaphor in Literary Machine Translation: Style, Creativity

10 KonText in Trilingual Studies—Supporting Phraseology

11 Voyant Tools’ Little Outing: How a Text Reading and

12 (Re)creating Equivalence of Stylistic Effect: A Corpus-Aided

13 The Experiment 237

14 Augmenting and Informing the Translation Process through

She is currently leading a Netherlands initiative for Education (NRO)

a member of the executive board of the Austrian Association of Literary

Andreas Van Cranenburgh is Assistant Professor at the University of

I.1 Computer-Assisted Literary Translation (CALT)—the

literariness at all by simply referring to ‘literary texts’ and assuming that

Another variation involves the attempt to make a distinction between

Chapter 1, this volume) discuss literariness and its ‘predictability’ based on

any given point in time, translation—including literary translation—is very

workshops at Swansea University2; the August 2019 Dublin Machine

• How does CALT affect the nature of literary translation as an activity

• How can these technologies be used in the translation of different lit-

I.2 Machine Translation—the State of the Art

Accordingly, once competitive neural network (NN)-based approaches to

In these latter two phases, the human translation is deemed to be ‘the’

I.3 Machine Translation of Literature—the State of the Art

total of over 100 million words of literary text—and 1,000 novels in

out independently at the same time, appearing at MT Summit 2019, where

outputs contain ‘markers’ that signify that their provenance is MT. He

I.4 Non-MT CALT—Research Directions

I.5 The Chapters

the focus shifts again to MT in the translation of poetry, widely assumed,

range of CALT tools, including corpora containing non-literary translations

Meldrum, Y. (2009). Translationese in Japanese literary translation. TTR:

Slessor, S. (2020). Tenacious technophobes or nascent technophiles? A survey of

July/Aug 2021. https://multilingual​.com​/issues​/july​-august​-2021​/translation​

The Automated and Post-

Antonio Toral, Andreas van Cranenburgh,

RQ1. Is it worthwhile building a literary-adapted MT system for a well-

1.2 Methodology and Experimental Setup

Table 1.1 Dataset sizes (words, on the source and target sides).

Dataset # # Sentences or # Words

2 This dataset (https://paracrawl​.eu/) contains around 89 million sentence pairs. A subset of

1.2.2 Pre- and Post-Processing

We separate words from adjacent punctuation symbols with scripts from

1.2.3 MT Evaluation Metrics

We use two automatic metrics to estimate the translation quality of the

1.2.4 MT System Building

We train several Transformer models (Vaswani et al., 2017) using dif-

on isolated sentences (one at a time) or on longer blocks made up of a few

a state-of-the-art open-source system trained on publicly available data,

MT system BLEU ▵% BLEU COMET ▵% Training time

DeepL 33.83 ― 0.512 ― ―

For more information about this series, please visit: www.routledge.com

I.1 Computer-Assisted Literary Translation (CALT)—the

July/Aug 2021. https://multilingual.com/issues/july-august-2021/translation

1.2 Methodology and Experimental Setup

Table 1.1 Dataset sizes (words, on the source and target sides).

2 This dataset (https://paracrawl.eu/) contains around 89 million sentence pairs. A subset of

1.2.2 Pre- and Post-Processing

1.2.3 MT Evaluation Metrics

1.2.4 MT System Building

1.5.1 S3Big vs S3BigDoc800: Sentence-Level vs Document-Level

Table 1.3 Distribution of translation issues in systems

1.5.2 Combined System vs DeepL

Table 1.4 Mean scores from the overall survey (N = 19).

Table 1.5 Mean scores for opening passages (N = 9).

Table 1.6 Mean scores for high literary passages

1.6 Conclusions and Future Work

1 Karin Tidbeck, Sylvain Neuvel, and Aljoscha Burchardt.

2.4 Data and Discussion

2.4.1 Temporal and Technical Effort

3.2 Literary Style and Translation

3.3 Oeser’s Style as a Translator and Post-Editor

2 See https://hanschristianoeser.wixsite.com/hcoeser for further details.

3.4 Keywords as a Method in Investigations of Style

Table 3.1 Basic for statistics for DeepL MT and Oeser PE.

4 www.deepl.com/translator (last accessed 30 September 2022).

Table 3.2 Positive keywords in Oeser PE compared to DeepL MT.