Human Issues in Translation Technology

Human Issues in Translation
Technology
Translation technologies are moulded by and impact upon humans in all sorts of
ways. This state-of-the-art volume looks at translation technologies from the point
of view of the human users – as trainee, professional or volunteer translators, or as
end users of translations produced by machines.
Covering technologies from machine translation to online collaborative
platforms, and practices from ‘traditional’ translation to crowdsourced translation
and subtitling, this volume takes a critical stance, questioning both utopian
and dystopian visions of translation technology. In eight chapters, the authors
propose ideas on how technologies can better serve translators and end users of
translations. The first four chapters explore how translators – in various contexts
and with widely differing profiles – use and feel about translation technologies
as they currently stand, while the second four chapters focus on the future: on
anticipating needs, identifying emerging possibilities, and defining interventions
that can help to shape translation practice and research.
Drawing on a range of theories from cognitive to social and psychological, and
with empirical evidence of what the technologization of the workplace means to
translators, Human Issues in Translation Technology is key reading for all those
involved in translation and technology, translation theory and translation research
methods.
Dorothy Kenny is Associate Professor in the School of Applied Language and

Intercultural Studies at Dublin City University, Ireland. She is the author of Lexis
and Creativity in Translation (Routledge, 2001), and served on the Executive
Council of the International Association for Translation and Intercultural Studies
from 2004 to 2016.
The IATIS Yearbook
Series editor: Jenny Williams
The International Association for Translation and Intercultural Studies (IATIS)

is a worldwide forum designed to enable scholars from different regional and
disciplinary backgrounds to debate issues pertinent to translation and other forms
of intercultural communication.
The series aims to promote and disseminate innovative research, rigorous
scholarship and critical thinking in all areas of translation studies and intercul-
tural communication.
Authorizing Translation
Michelle Woods
Human Issues in Translation Technology

Dorothy Kenny
Human Issues in Translation
Technology
Edited by Dorothy Kenny

First published 2017
by Routledge
2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN
and by Routledge
711 Third Avenue, New York, NY 10017
Routledge is an imprint of the Taylor & Francis Group, an informa business
 2017 International Association for Translation and Intercultural Studies / IATIS
The right of Dorothy Kenny to be identified as the author of the editorial
material, and of the authors for their individual chapters, has been asserted
in accordance with sections 77 and 78 of the Copyright, Designs and Patents
Act 1988.
All rights reserved. No part of this book may be reprinted or reproduced or
utilised in any form or by any electronic, mechanical, or other means, now
known or hereafter invented, including photocopying and recording, or in any
information storage or retrieval system, without permission in writing from
the publishers.
Trademark notice: Product or corporate names may be trademarks or
registered trademarks, and are used only for identification and explanation
without intent to infringe.
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Cataloging-in-Publication Data
Names: Kenny, Dorothy, author.
Title: Human issues in translation technology / by Dorothy Kenny.
Description: Milton Park, Abingdon, Oxon ; New York, NY : Routledge,
[2017] | Series: The IATIS Yearbook | Includes bibliographical
references and index.
Identifiers: LCCN 2016020421| ISBN 9781138123298 (hardback) |
ISBN 9781315648934 (ebook)
Subjects: LCSH: Translating services. | Translating and interpreting—
Technological innovations. | Translating and interpreting—Data
processing.
Classification: LCC P306.97.T73 K36 2017 | DDC 418/.020284—dc23
LC record available at https://lccn.loc.gov/2016020421
ISBN: 978-1-138-12329-8 (hbk)

ISBN: 978-1-315-64893-4 (ebk)
Typeset in Times New Roman

by Swales & Willis Ltd, Exeter, Devon, UK
For Professor Leslie Davis, without whom none of this would
have happened.
Contents
List of f igures ix
List of tables xi
List of contributors xii
Series editor’s preface xvi
Acknowledgements xvii
Introduction 1
DOROTHY KENNY
1 Love letters or hate mail? Translators’ technology acceptance

in the light of their emotional narratives 8
KAISA KOSKINEN AND MINNA RUOKONEN
2 Deconstructing translation crowdsourcing with the case of a

Facebook initiative: A translation network of engineered
autonomy and trust? 25
MINAKO O’HAGAN
3 ‘I can’t get no satisfaction!’ Should we blame translation

technologies or shifting business practices? 45
MATTHIEU LEBLANC
4 How do translators use web resources? Evidence from

the performance of English–Chinese translators 63
VINCENT X. WANG AND LILY LIM
5 Translators’ needs and preferences in the design of

specialized termino-lexicographic tools 80
ALEJANDRO GARCÍA-ARAGÓN AND CLARA INÉS LÓPEZ-RODRÍGUEZ
viii Contents
6 Assessing user interface needs of post-editors of
machine translation 109
JOSS MOORKENS AND SHARON O’BRIEN
7 Issues in human and automatic translation

quality assessment 131
STEPHEN DOHERTY
8 Cn U read ths? The reception of txt language in subtitling 149

ALINA SECARĂ
Index 171
F igures
1.1 Usability and positive affects 20

1.2 Usability and negative affects 21
2.1 A potential network of actors linked to the FTC platform 34
2.2 Facebook Translations platform configuration 35
5.1 Conceptual network for the concept acid rain / lluvia ácida,
from EcoLexicon 87
5.2 Conceptual map for acid depositions / deposiciones ácidas,
in which acid rain is included, generated with CmapTools 88
5.3 Statements and results of the user evaluation of the questionnaire 89
5.4 Answers for and against the division of knowledge into
separate layers for experts, semi-experts and laypeople 93
5.5 Preferred mediums in a specialized dictionary 94
5.6 Choices on the possible roles of conceptual networks such as
that for acid rain in Figure 5.1 95
5.7 Choices on the possible roles of concept maps such as that for
acid depositions in Figure 5.2 96
5.8 Choices on knowledge representation techniques 97
5.9 User query routes in MeteoTrad according to
termino-lexicographic relevance for translators 99
5.10 Searching for wet acid deposition in MeteoTrad:
the four access routes 100
5.11 Excerpt from the route ‘Equivalents’ for wet
acid deposition in Spanish 100
5.12 Examples of termino-lexicographically relevant linking to
related results in other routes 100
5.13 Mean and standard deviation for each statement about the
MeteoTrad homepage 101
Equivalents route 102
5.15 Mean and standard deviation for each statement about
the Usage route 102
Definitions route 103
x List of f igures
Knowledge route 103
6.1 Survey question with radio button responses 114
6.2 Participants’ age range 115
6.3 Tools used for translation and post-editing 118
6.4 TM fuzzy match thresholds below which an MT match is favoured 120
8.1 Example of subtitles in C1, first and second parts of the file 156
8.2 Example of subtitles in C2, first and second parts of the file 157
8.3 Regressions, group C2, participant 1 160
8.4 Histogram for regressions in the two Experiment groups 161
8.5 Gaze path of C2 participant, first part of the file 164
8.6 Gaze path of C2 participant, second part of the file 164
Tables
1.1 All letters by respondents 11

1.2 Technology letters by category and respondent 14
1.3 Letters commenting on translation-specific tools 14
1.4 Translators’ vs students’ comments on translation-specific
technology 16
1.5 Love and break-up letters and their attitudes towards change 16
1.6 Work experience vs positive and negative affects
towards technology 17
4.1 Comparison of the number of web searches by the two groups of
translators 68
4.2 Comparison of the use of web resources by the two groups of
translators 68
4.3 Number of searches for key expressions by the junior and
senior groups 69
4.4 Websites visited by the junior and senior groups to search for
key expressions 69
4.5 Translations of capped by the two groups of translators 75
5.1 Parallelisms between termino-lexicographic functions,
translation sub-competences and dictionary items 84
5.2 Respondents’ preferences on number of languages, type of
dictionary and subject field 93
6.1 Participants’ target languages 116
6.2 Keyboard shortcuts requested 118
6.3 Language-specific keyboard shortcuts requested 119
6.4 Interviewee profiles 123
7.1 Thresholds for inter-rater reliability kappa scores 140
8.1 Word and character count in subtitling files for C1 and C2 154
8.2 Reduction in character count in C2 154
8.3 txt lingo characteristics in the first and second parts of the
C2 subtitling file 156
8.4 Regressions t-test results in Control 160
8.5 Regressions t-test results in Experiment 162
8.6 Fixation count t-test results in Control 162
8.7 Fixation count t-test results in Experiment 162
Contributors
Stephen Doherty is a senior lecturer in the School of Humanities and Languages at

the University of New South Wales, Australia. A doctoral graduate of Dublin City
University, Ireland, he researches language and cognition using eye-tracking in
areas of translation technologies and translation process studies. He is author of
Eye-Tracking in Translation (Routledge, forthcoming) and co-editor of Human
and Machine Translation Quality and Evaluation (Springer, forthcoming),
and has published numerous book chapters and articles in journals such as the
International Journal of Communication, Perspectives: Studies in Translatology,
The Interpreter and Translator Trainer and Machine Translation.
Alejandro García-Aragón is a doctoral graduate and member of the LexiCon
research group at the University of Granada, Spain. His major fields of research
include lexicography, terminology, specialized translation, knowledge
representation and Modern Greek studies. He has authored and co-authored
several book chapters and international conference presentations, as well as
articles in Terminology, the Journal of Specialised Translation, Trans-Kom,
the International Journal of Information Technologies and Knowledge, Major
Trends in Theoretical and Applied Linguistics, the Proceedings of the Hellenic
Society for Terminology and Poliphilos.
Dorothy Kenny is Associate Professor in the School of Applied Language
and Intercultural Studies at Dublin City University, Ireland, where she
lectures in translation technology, terminology and corpus linguistics. Her
publications include: Lexis and Creativity in Translation: A Corpus-Based
Study (Routledge, 2001) and the co-edited volumes Unity in Diversity: Current
Trends in Translation Studies (Routledge, 1998) and Across Boundaries:
International Perspectives on Translation Studies (Cambridge Scholars, 2007).
She has authored numerous refereed articles and book chapters on corpus-
based translation studies, computer-aided translation, translator training and
translation theory. She is a member of the Centre for Translation and Textual
Studies at Dublin City University.
Kaisa Koskinen is Professor of Translation Studies at the University of
Tampere, Finland. She is the author of Beyond Ambivalence: Postmodernity
List of contributors xiii
and the Ethics of Translation (PhD thesis, University of Tampere 2000) and
Translating Institutions: An Ethnographic Study of EU Translation (Routledge,
2008). She is co-author (with Tytti Suojanen and Tiina Tuominen) of User-
Centered Translation (Routledge, 2015) and co-editor (with Tuija Kinnunen)
of Translators’ Agency (Tampere University Press, 2010). Her current research
interests include usability and translation, translation and affect, literary
retranslations in Finland and the city of Tampere as a translational space.
Matthieu LeBlanc is Full Professor in the Department of Translation and
Modern Languages and Vice Dean of the Faculty of Arts and Social Sciences
at the Université de Moncton, New Brunswick, Canada. A former professional
translator, he holds a PhD in sociolinguistics. His current research focuses on the
translator’s status and translation practices in an increasingly automated working
environment, as well as on language policies and the role of translation in bilingual
settings. His work has been published in journals such as TTR, Meta, Langage
et société, Minorités linguistiques et société and the International Journal for
Translation and Interpreting Research. He is co-editor of La francophonie en
Acadie : dynamiques sociales et langagières (Prise de Parole, 2014).
Lily Lim has taught translation and interpreting at the Macau Polytechnic
Institute since 2002, where she is now assistant coordinator for the Chinese–
English Translation Programme. She holds a Master’s and a PhD in applied
linguistics from the University of Queensland, Australia, and a Masters in
software engineering from the University of Macau. She is both a practitioner
and a trainer of conference interpreters, and holds a Certificate in Chinese–
Portuguese Conference Interpreting from the European Commission’s
Directorate-General for Interpretation (DG SCIC). Her research focuses on
computer-assisted interpreter training and effective teaching approaches for
translators and interpreters. She has published papers in ReCALL, Bable and
The Interpreter and Translator Trainer, as well as book chapters with Rodopi,
Cambridge Scholars and Bookman Books.
Clara Inés López-Rodríguez is tenured Professor in the Department of Translation
and Interpreting of the University of Granada, Spain, where she teaches scientific
and technical translation, and multimedia translation (subtitling and localization).
She holds a PhD from the University of Granada and belongs to the LexiCon
research group. Her current research deals with scientific translation, terminology,
and the application of corpus linguistics to terminology and translation. She
has published in several journals included in the European Reference Index for
the Humanities, such as Terminology, Meta, TTR, the Journal of Specialised
Translation, New Voices in Translation Studies, Lebende Sprachen, Perspectives:
Studies in Translatology, the Belgian Journal of Linguistics and Sendebar.
Joss Moorkens is a lecturer in translation technology in the School of Applied
Language and Intercultural Studies in Dublin City University, Ireland, and a
member of both the ADAPT Centre, an industry-academia partnership, and
the Centre for Translation and Textual Studies at Dublin City University.
xiv List of contributors
He is co-editor of Human and Machine Translation Quality and Evaluation
(Springer, forthcoming), and has authored several journal articles, book
chapters and conference papers on topics such as translation memory, user
evaluation of translation technology (especially machine translation), and
ethical considerations in translation technology in relation to both machine
learning and professional practice. Within ADAPT, he has contributed to the
development of translation tools for both desktop and mobile.
Sharon O’Brien is a senior lecturer in the School of Applied Language and
Intercultural Studies, Dublin City University, Ireland. Her research focuses
on the interaction between translators and technology, post-editing, cognitive
aspects of translation, quality assessment, research methods (including
eye-tracking and keyboard logging), localization and content authoring.
She is Director of the Centre for Translation and Textual Studies at Dublin
City University and is a funded researcher in the ADAPT Centre. Her
recent publications include Research Methodologies in Translation Studies
(Routledge, 2013, with Gabriela Saldanha), and the co-edited volumes
Interdisciplinarity in Translation and Interpreting Process Research (John
Benjamins, 2015) and Post-Editing of Machine Translation: Processes and
Applications (Cambridge Scholars, 2014), as well as numerous articles in
journals such as Machine Translation, Target and Translation Spaces.
Minako O’Hagan has been Associate Professor in the School of Languages,
Cultures and Linguistics at the University of Auckland, New Zealand, since
September 2016, having previously worked at Dublin City University, where
she lectured in translation technology, multimedia translation, terminology and
Japanese translation. She has research specialisms in translation technology,
video-game localization and collaborative translation, and her recent publications
include Game Localization: Translating for the Global Digital Entertainment
Industry (Routledge, 2013, co-authored with Carmen Mangiron), Fun for All:
Translation and Accessibility Practices in Video Games (Peter Lang, 2014,
co-edited), ‘Translation as a Social Activity: Community Translation 2.0’, a
special edition of Linguistica Antverpiensia (2011), and numerous book chapters
and articles in journals such as the International Journal of Communication,
Translation Spaces, Translation Studies and Multilingua.
Minna Ruokonen is a senior lecturer in English language and translation at
the University of Eastern Finland in Joensuu. She is currently investigating
the status of translation as a profession – particularly translators’ status
perceptions and actions for changing their status. Her recent publications
include an overview of research on translator status (2013) and two
reports on Finnish translation students’ views of translator status. Her PhD
dissertation (2010) focused on the translation of literary allusions in the
Finland of the 1940s and the 1980s. She has also co-edited two issues of
MikaEL (Electronic Proceedings of the KäTu Symposium on Translation and
Interpreting Studies).
List of contributors xv
Alina Secară is a lecturer in translation studies and Programme Manager for the
MA in Audiovisual Translation Studies at the University of Leeds, United
Kingdom. Trained in translation and certified as a theatre captioner, she
delivers audiovisual translation and computer-assisted translation training
courses to MA students and other professionals, including staff of European
Union and United Nations agencies. She engages in research with academic
and industrial partners throughout Europe. She is currently working with
the Translators without Borders European Refugee Crisis Response Team
to design and deliver bespoke translation training for crisis situations, and is
part of the EU-funded DigiLing project (2016–19) set up to create e-learning
resources in the area of digital linguistics.
Vincent X. Wang is Professor in the Department of English at the University
of Macau, where he teaches translation studies and comparative linguistics.
He holds a PhD in applied linguistics from the University of Queensland and
his monograph Making Requests by Chinese EFL Learners was published by
John Benjamins in 2011. His current research focuses on contrasble linguistic
investigations of Chinese and English using both comparable and parallel
corpora. His work in translation studies and applied linguistics has appeared
in journals such as Target, Translation Watch Quarterly, the Journal of
Language, Literature and Culture and TESOL in Context. He practises as a
translator and conference interpreter in Macau.
Series editor’s preface
The International Association for Translation and Intercultural Studies (IATIS)

is a worldwide forum designed to enable scholars from different regional and
disciplinary backgrounds to debate issues pertinent to translation and other forms
of intercultural communication.
By bringing together the disciplines of translation studies and intercultural
studies under one umbrella, the Association facilitates a dialogue that encompasses,
but also goes beyond, traditional academic boundaries. IATIS is convinced that
the intellectual challenges posed by globalization and multiculturalism can be met
only by a genuinely interdisciplinary approach.
The Association pursues this objective through the organization of a triennial
international conference and regional workshops, and the creation of web-based
resources, as well as the publication of a Yearbook and the online journal New
Voices in Translation Studies.
The Yearbook, which addresses not only IATIS members, but also the
international scholarly community, is a key publication for the Association.
Each Yearbook is devoted to a topical theme and guest-edited by an expert in
the field. Previous Yearbooks have covered topics such as translation studies in
Africa (2009), cognitive explorations of translation (2010) and self-translation
(2013). The series aims to promote and disseminate innovative research, rigorous
scholarship and critical thinking in all areas of intercultural communication.
The 2016 IATIS Yearbook, edited by Dorothy Kenny, is devoted to the human
interaction with translation technology – a topic that has received little attention
from translation studies scholars to date. The issues raised go beyond the individual
experience of using particular technologies to address universal questions of the
relationship between human beings and the tools we use.
Jenny Williams
Chair, IATIS Publications Committee
Dublin City University, Ireland
Acknowledgements
I am deeply indebted to all of the contributors and anonymous peer reviewers

whose hard work has made this volume possible. Thank you. I would also like
to thank the series editor, Professor Jenny Williams, as well as Louisa Semlyen
and Laura Sandford at Routledge, for their patience and support over the long
gestation of this project. The ongoing support of the International Association for
Translation and Intercultural Studies is also gratefully acknowledged. On a more
personal note, I would like to thank my family – Graham, Niamh and Rowan –
for their unflinching encouragement, and Caroline Whitston and Paula Smith, for
all of the practical ways in which they helped me to get this book over the line.
Finally, the reader may notice that many of the contributors to this volume have,
at some time or another, been associated with the School of Applied Language
and Intercultural Studies (SALIS) at Dublin City University, Ireland. This is no
coincidence. Since its very beginnings in the early 1980s, SALIS has been a place
that has fostered critical and practical engagement with translation technologies of
all sorts. This is in no small part owing to the prescience and intellectual openness
of its first Head of School, Professor Leslie Davis, to whom this book is dedicated.
Dorothy Kenny
July 2016
Every effort has been made to contact copyright-holders. Please advise the publisher
of any errors or omissions and these will be corrected in subsequent editions.
Introduction
Dorothy Kenny
Human Issues in Translation Technology is a coy title for a book. It seems

simultaneously slow to reveal exactly what the book is about and reluctant to
commit to any particular methodological approach. Even if we live in an age in
which almost anything we write, say, click or swipe (in digital environments, at
least) can be harvested by one automated system to be shared immediately with
another automated system and humans appear to be temporarily cut out of the
loop, humans are surely always present as the originators of content, the ultimate
targets of adverts, the users of analytics, the developers of algorithms, the data
capitalists who reap the benefits. Translation technologies are no different. They
are moulded by and impact upon humans in all sorts of ways, and it would not
be difficult to claim that all issues in translation technology are ‘human’ issues.
Rather than trying to isolate a discrete set of specific questions within translation
technology, however, the title of this book is intended to be suggestive of a
perspective – one that privileges humans, no matter what their role in the vast
area that is associated with translation technologies.
That such a perspective is needed should come as no surprise to those interested
in studies of technology: it has been observed, for example, that the history of
technology has tended to focus on invention and innovation rather than on how
technologies are actually used (Edgerton 2011). A ‘pro-innovation bias’ is also
manifest in the sometimes implicit understanding in much of the literature that
innovation is self-evidently good (Sveiby et al. 2012) – an understanding that can,
in turn, lead to an insufficient questioning of the need for and the effects on society
of given technologies. Once the focus is switched to use, however, the desires and
abilities of – and impacts on – human beings come into play and many ‘older’
technologies appear to be longer-lived than previously assumed (Edgerton 2011).
A pro-innovation bias can also be associated with an overemphasis on the
inherent characteristics of new technologies, on their novel functions and
technical features, at the expense of any consideration of how those technologies
emerge in, fit into or change their sociotechnical environments. These latter,
broader, questions have, of course, been tackled in the field known as science
and technology studies (STS). Without wishing to oversimplify, scholars in
STS generally reject the idea of ‘technoneutrality’ – to use Tehranian’s (1990)
term. This is a position held by commentators who believe ‘that technologies in
2 Dorothy Kenny
themselves are entirely neutral, that they don’t take sides, and that in the right
hands they can do marvels’ (Morozov 2013: 169). Technoneutrals tend not to
focus on the agendas of the creators of technologies or on the contexts of use of the
technologies in question. In comparison, technostructuralists – the other term in
Tehranian’s (1990) dichotomy – believe that the impact of technologies ‘is always
mediated through the institutional arrangements and social forces, of which they
are an integral part’ (Tehranian 1990: 5). For technostructuralists, the impact of a
technology is believed neither to flow from its inherent characteristics nor to be
neutral; rather, it depends on the context, and its impact is studied by ‘analyzing
how particular aspects of a given technology . . . might restructure political and
social relations, introducing entirely new classes of actors into the game’ (Morozov
2013: 170). Technostructuralists, while not necessarily pessimistic, are also more
likely than technoneutrals to study the unanticipated, and sometimes undesirable,
consequences of new technologies (Morozov 2013: 171). While technoneutrality
is associated with blindness to context, inadequate understanding of social and
institutional forces, and impoverished ethical reasoning, technostructuralism
and STS in general offer a much richer way in which to view the impact of
technologies, including translation technologies.
Given the potential of STS in translation technology, the reader might
justifiably ask why this is not a volume on ‘STS-inspired studies of translation
technology’, or ‘social-constructivist approaches to translation technology’, or
even ‘technostructuralist approaches to translation technology’. Why, in other
words, does the book not commit to this broad theoretical framework and its
attendant methodologies? The answer is a pragmatic one: there are, to date, almost
no studies in translation technology that fit the bill. That said, there are, in Olohan’s
(2011) work, the beginnings of an approach to translation technology that is
inspired by STS: Olohan draws on Pickering (1995) to develop an understanding
of how translators interact with a new version of a popular translation memory
(TM) tool, offering a model of how the translation-technological ‘tunes into’ the
social and vice versa. Olohan’s (2011) approach has directly inspired one of the
chapters in this volume – that by Vincent Wang and Lily Lim – suggesting that
ideas from Pickering’s sociology of technology are gaining traction in translation
studies. Meanwhile, a second chapter, by Minako O’Hagan, draws on yet another
doyen of constructivist approaches to science and technology, Bruno Latour
(1987, 2005), to examine translation practices at Facebook. In another recent
study, O’Hagan (2016) draws on Grimes and Feenberg’s (2013) critical theory of
technology (CTT) to conduct a related analysis of both Facebook and Wikipedia
translation. It should also be noted that other work, such as that by Cronin (2013),
Garcia (2007), Kenny (2011) and Moorkens et al. (2016), also points towards an
engagement with translation technology that is critical and takes economic factors
into account, although none of these sources is explicitly inspired by STS or CTT.
Even if many of the chapters in this book do not explicitly tap into STS
scholarship, they share with various scholars in STS a concern with: contexts of use
of technologies; users and their responses to various technologies; the involvement
of users in the creation of technologies; and the social, economic and ethical
Introduction 3
import of the deployment of those technologies. Their points of reference may
come from information systems, usability studies, psychology, reading studies,
audiovisual translation studies, machine translation (MT) or cognitive science,
but they all have humans as their central concern, and where they do home in on
technical features, it is specifically to find ways of better serving human users,
right from the design phase. For despite claims about the democratizing nature
of digital technologies – and specifically the idea that ‘democratic rationalization
of technology occurs when the general public intervenes in the design of
technologies based on their user experience’ (O’Hagan 2016: 934, referring to
Grimes and Feenberg 2013) – professional and non-professional translators alike
do not seem to have had much impact to date on the design of the tools that they
are called upon to use. The projects reported on in this volume by Moorkens and
O’Brien, on one the hand, and García-Aragón and López-Rodríguez, on the other,
the ultimate aim of which is to create interfaces and resources that will better meet
translators’ needs, thus represent very practical interventions in the field, as well
as specific answers to the question that Latour (2003: 45) argues constructivist
approaches to technology should be asking – namely: ‘How can it be built better?’
Not surprisingly, perhaps, given the foregoing comments on the longevity
of certain technologies, the chapters in this book cover both long-established
technologies and more recent innovations. García-Aragón and López-Rodríguez
are thus concerned with lexical resources and knowledge bases, continuing in a
tradition that goes back centuries, and while they recognize the profound changes
that digital technologies have brought to translation, they note that one thing that
has not changed is that translators still ‘spend more than half of their translation
time consulting reference works’. Likewise, dictionaries – again, albeit online
dictionaries – are heavily used by the translators in Wang and Lim’s study. Other
authors focus on translation memory (LeBlanc; Moorkens and O’Brien) or machine
translation (Moorkens and O’Brien; Doherty) – technologies that saw their first
concrete realizations in the 1990s and 1950s, respectively – while Koskinen and
Ruokonen’s research design is flexible enough to allow participants to focus on
any technology of their choice, from search engines to machine translation (both
of which are also broached by Wang and Lim).
Nor do any of the authors who have contributed to this book put forward a
notion of neutral, value-free technology. And, like the translators with whom they
work, some authors are all too aware that technologies can be easily conscripted
into wider projects of economic control, with LeBlanc (this volume) arguing,
for example, that ‘what seems to have unsettled translators the most is not so
much the tool’s inherent design . . . but more so the shifts in administrative and
business practices that have, in some cases, accompanied [translation memory]
implementation’.
The contributors to this volume also share with STS a strong interest in
empirical studies, although there are no classic case studies in the STS tradition (cf.
Bijker 1995); rather, the methods applied here run the gamut from the elicitation
of narrative-performative data (in Koskinen and Ruokonen’s contribution), to
ethnography and netnography (in LeBlanc and O’Hagan’s respective chapters),
4 Dorothy Kenny
experimental designs involving observation and retrospective interviews of
participants performing timed tasks (Wang and Lim; Secară, who uses eye-
tracking), large-scale surveys (García-Aragón and López-Rodríguez; Moorkens
and O’Brien, who also use interviews), and meta-analysis with a helping of
discourse analysis (Doherty). As well as covering broad methodological terrain,
the book has a wide geographic spread, with contributors from Australia, Canada,
Finland, Ireland, Macau, New Zealand, Spain and the United Kingdom.
This book is broadly structured as follows: the first four chapters are about
how translators – in various contexts and with widely differing profiles – use and
feel about translation technologies as they currently stand, while the second four
chapters are more about what might be the case in the future. These latter chapters
focus on anticipating needs, identifying emerging possibilities, and defining
interventions that might help to shape translation practice and research.
In Chapter 1, ‘Love letters or hate mail? Translators’ technology acceptance
in the light of their emotional narratives’, Kaisa Koskinen and Minna Ruokonen
investigate how translators connect emotionally with technologies in their work.
Drawing on a method borrowed from usability studies, they ask participants
to write a short love letter or break-up letter to a tool, application or aspect of
work of their choice, eliciting more than 100 letters dealing with technologies
of various kinds used by translators. Their findings lead them to question the
commonly held assumption that translators tend to have a negative mindset when
it comes to technology and neither do they find evidence of a generation gap in the
emotional narratives that they elicit – but their results do hint at the possibility that
user involvement in the development of a technology can be linked to positive
emotions towards that technology.
In Chapter 2, ‘Deconstructing translation crowdsourcing with the case of a
Facebook initiative: A translation network of engineered autonomy and trust?’,
Minako O’Hagan tackles a controversial technology-mediated development in
translation studies: crowdsourcing. She uses Latour’s (2005) actor–network
theory (ANT) to highlight the ways in which the technological environment
helps to engineer autonomy and trust among the actors in the network. Her
study suggests that the technological platform used in Facebook Translation
Crowdsourcing (FTC) may initially satisfy volunteer translators’ need for a
sense of autonomy, but ultimately lessens their sense of autonomy by limiting
their control of the technical environment. Likewise, the volunteers’ trust in the
organizers of the initiative can diminish over time if the latter are not responsive
to volunteers’ needs.
In Chapter 3, ‘ “I can’t get no satisfaction”: Should we blame translation
technologies or shifting business practices?’, Matthieu LeBlanc addresses
similar themes of autonomy and affect, but in a different context to the preceding
authors. He draws on data collected through ethnographic observation in three
Canadian translation service providers (TSPs) to explore the shifts in business
and administrative practices that have accompanied their implementation of
TM technology. While data from two of the firms reveal concerns about
professional satisfaction, status, loss of autonomy, personal motivation and
Introduction 5
excessive automation of workflows, such concerns are not as common in the
third firm, in which TM is used as a simple translation aid, with no link to
productivity requirements and no enforced recycling.
In Chapter 4, ‘How do translators use web resources? Evidence from the
performance of English–Chinese translators’, Vincent Wang and Lily Lim
investigate how two groups of translators in Macau use resources such as online
bilingual dictionaries (often derived from corpora) and machine translation to
complete a timed translation task. Unlike LeBlanc’s study, Wang and Lim’s
research is not conducted in the workplace; instead, the authors opt for an
experimental research design, perhaps reflecting the very early stage of this kind of
translation-process-oriented research in the Chinese-speaking translation studies
community. Wang and Lim find notable differences in search behaviour between
junior (student) and senior translators, and attempt to explain their observations
using insights from Pickering (1995) and Pym (2011).
In Chapter 5, ‘Translators’ needs and preferences in the design of specialized
termino-lexicographic tools’, Alejandro García-Aragón and Clara Inés López-
Rodríguez investigate the information needs and preferences of translators when
using lexicographical tools and knowledge bases. They draw on the results of an
online survey carried out among 201 professional and trainee translators, and on
the subsequent experience of creating MeteoTrad, a specialized lexicographical
prototype that integrates insights from frame-based terminology (Faber 2012) and
the function theory of lexicography (Bergenholtz and Tarp 2010), and attempts
to reproduce the cognitive and communicative routes followed by translators
when performing specialized translation tasks. The analysis focuses specifically
on translators’ preferences in relation to traditional and new lexicographical
resources, the way in which they interact with these resources, and how they use
conceptual, linguistic and visual information.
In Chapter 6, ‘Assessing user interface needs of post-editors of machine
translation’, Joss Moorkens and Sharon O’Brien likewise delve into what users
of specific translation technologies want from those technologies. They present
the results of an extensive survey, with 403 respondents, and a series of follow-up
interviews with professional translators and post-editors, in which participants
discuss features and functions for their ideal translation and machine translation
post-editing user interface (UI). Participants stress the need for a simple and
customizable UI that supports multilingual formatting and follows established
translation editor conventions, in addition to some functionality specific to
post-editing. The survey and interview responses lead the authors to draw up
specifications for a UI that better supports the post-editing task.
In Chapter 7, Stephen Doherty addresses ‘Issues in human and automatic
translation quality assessment’, focusing on the decisions humans make (or do
not make) in designing translation evaluations, whether those evaluations will
ultimately be carried out by humans or machines. These decisions have to do,
among other things, with the human-produced materials used in translation
quality assessment (TQA), the selection of people to act as human evaluators,
the instructions given to those people and the subjective human decisions that
6 Dorothy Kenny
underlie automatic evaluations. Ultimately, Doherty’s research is concerned
with improving the methodological rigour of TQA, working on a meta level to
suggest ways in which the validity and reliability of empirical studies in this area
can be enhanced.
Finally, in Chapter 8, ‘Can U read ths? The reception of txt language in
subtitling’, Alina Secară is also concerned with changing practices. Having
observed continued uniformity in mainstream subtitle creation, despite the
proliferation of non-standard orthography in, for example, fan subtitling and
texting/instant messaging, she asks whether consumers of audiovisual products
would tolerate ‘creative’ subtitles and investigates the extent to which their
reading of such subtitles differs from their reading of traditional subtitles. She thus
reports on an eye-tracking experiment designed to elicit data on readers’ physical
responses to creative vs traditional subtitles and supplements her findings with
questionnaire data on consumer tolerance for, and comprehension of, creative txt
spellings in interlingual subtitles.
Despite its breadth, one has the sense that we are merely scratching the surface
of a vast area in this book. While we are well served in translation studies by (text)
books that explain how translation technologies work (e.g. Bowker 2002; Quah
2006; Koehn 2010; O’Hagan and Mangiron 2013; Roturier 2015, to name but a
few), and we can now assume that most of the technologies involved are fairly
familiar to the interested reader and thus do not require elaborate explanation
here, it has already been acknowledged that broader STS-inspired approaches to
translation technology are still few and far between. It is hoped that, with volumes
such as this, we are on the cusp of a refinement of the ‘technological turn’ in
translation studies – one that will see more critical humanities- and social-science-
inspired research into translation technologies.
References
Bergenholtz, H., and S. Tarp. 2010. ‘Lexicography or Terminography? The Lexicographer’s
Point of View’. In P. A. Fuertes Olivera (ed.). Specialized Dictionaries for Learners.
Berlin/New York: De Gruyter, 27–36.
Bijker, W. E. 1995. Of Bicycles, Bakelites and Bulbs: Towards a Theory of Sociotechnical
Change. Cambridge, MA: MIT Press.
Bowker, L. 2002. Computer-Assisted Translation Technology: A Practical Introduction.
Ottawa, ON: University of Ottawa Press.
Cronin, M. 2013. Translation in the Digital Age. London/New York: Routledge.
Edgerton, D. 2011. The Shock of the Old: Technology and Global History since 1900.
Oxford: Oxford University Press.
Faber, P. (ed.). 2012. A Cognitive Linguistics View of Terminology and Specialized
Language. Berlin/Boston: Mouton de Gruyter.
Garcia, I. 2007. ‘Power Shifts in Web-Based Translation Memory’. Machine Translation
21(1): 55–68.
Grimes, S. M., and A. Feenberg. 2013. ‘Critical Theory of Technology’. In C. Jewitt,
B. Brown and S. Price (eds). SAGE Handbook of Digital Technology Research.
London: Sage, 121–30.
Introduction 7
Kenny, D. 2011. ‘The Ethics of Machine Translation’. In Proceedings of the 11th New Zealand
Society of Translators and Interpreters Annual Conference. Auckland: NZSTI, 121–31.
Koehn, P. 2010. Statistical Machine Translation. Cambridge: Cambridge University Press.
Latour, B. 1987. Science in Action. Cambridge, MA: Harvard University Press.
Latour, B. 2003. ‘The Promises of Constructivism’. In D. Idhe and E. Selinger (eds).
Chasing Technoscience: Matrix of Materiality. Bloomington, IN: Indiana University
Press, 27–46.
Latour, B. 2005. Reassembling the Social: An Introduction to Actor–Network Theory.
Oxford: Oxford University Press.
Moorkens, J., D. Lewis, W. Reijers, E. Vanmassenhove and A. Way. 2016. ‘Translation
Resources and Translator Disempowerment’. In Proceedings of ETHI-CA² 2016: ETHics
in Corpus Collection, Annotation and Application, 49–53. Available at: http://www.
lrec-conf.org/proceedings/lrec2016/workshops/LREC2016Workshop-ETHICA2_
Proceedings.pdf [accessed 12 March 2016].
Morozov, E. 2013. To Save Everything, Click Here. London: Allen Lane.
O’Hagan, M. 2016. ‘Massively Open Translation: Unpacking the Relationship
between Technology and Translation in the 21st Century’. International Journal of
Communication 10: 929–46.
O’Hagan, M., and C. Mangiron. 2013. Game Localization: Translating for the Global
Digital Entertainment Industry. Amsterdam/Philadelphia: John Benjamins.
Olohan, M. 2011. ‘Translators and Translation Technology: The Dance of Agency’.
Translation Studies 4(3): 342–57.
Pickering, A. 1995. The Mangle of Practice. Chicago, IL: University of Chicago Press.
Pym, A. 2011. ‘What Technology Does to Translating’. International Journal for
Translation and Interpreting Research 3(1): 1–9.
Quah, C. K. 2006. Translation and Technology. New York: Palgrave Macmillan.
Roturier, J. 2015. Localizing Apps: A Practical Guide for Translators and Translation
Students. London/New York: Routledge.
Sveiby, K.-E., P. Gripenberg and B. Segercrantz (eds). 2012. Challenging the Innovation
Paradigm. London/New York: Routledge.
Tehranian, M. 1990. Technologies of Power: Information Machines and Democratic
Purposes. New York: Ablex.
1 Love letters or hate mail?
Translators’ technology acceptance in the
light of their emotional narratives
Kaisa Koskinen and Minna Ruokonen
Introduction
In the past three decades, translation has rapidly shifted from a predominantly
humanist profession to an increasingly technology-driven practice: translators
have had to come to grips with such new technologies as word processing in the
1980s, translation memory (TM) tools in the 1990s, and, most recently, human-
aided machine translation (MT), speech-recognition software and others. These
technologies have repeatedly changed the face of the profession, challenging
those working in this field to readjust their thinking and their work practices, and
to constantly adopt new technology.
In the translation industry, new tools are introduced for rational reasons
(speed, efficiency, accuracy), but the emotional side of new technology, and
the effects of good or bad user experiences, are also relevant for explaining
translators’ technology acceptance processes and for understanding the social
effects of technologization on the translation profession. It has been observed
that understanding and managing the emotions of different professional groups
is relevant for those in charge of introducing new technologies in the workplace
(e.g. Venkatesh and Bala 2008). It logically follows that, in introducing new
translation technology, understanding translators’ emotions is highly relevant – in
particular, given the general assumptions of translators’ technology-averseness
and reluctance in accepting new tools (e.g., Drugan 2013: 24). Such assumptions,
however, remain largely intuitive; the results that might be used to support or
disprove this argument are either not very recent (see survey of literature in
Dillon and Fraser 2006: 68) or are, by necessity, limited to a particular context
(e.g. LeBlanc 2013). Interestingly, LeBlanc’s 50+ interviews at three Canadian
translation agencies indicate that translators would not be opposed to new
technologies (LeBlanc 2013: 10). Some evidence for or against technology may
be explained by generational differences in attitudes, as found by Dillon and
Fraser (2006: 73–5). There may also be differences in acceptance of information
technology (IT) in general and translation-specific tools in particular (Fulford and
Granell-Zahra 2005: 9–10). All in all, in translator–computer interaction (O’Brien
2012), translators’ emotions and affects are still a fairly under-researched area.1
This chapter looks at translators’ emotional narratives of the professional
practice of translation, with particular emphasis on the role of technology
Love letters or hate mail? 9
in translators’ work. It reports the findings of an exploratory project in which
participants were asked to write a short ‘love letter/break-up letter’ to the tool,
application or aspect of work of their choice. This method comes from usability
research, where it is used to study how people emotionally connect with devices
and objects (Hanington and Martin 2012: 114), and it also proved useful for
researching translators’ emotional attachments.
During spring 2014, a total of 148 letters were collected from 102 res-
pondents (see ‘Research methodology and the respondents’). The data allows
for a variety of analytical approaches. In this chapter, the focus is on attitudes
towards technology. Of the 148 letters, 106 either focus primarily on technology
or comment on it, and we use this subset as our data in the present study (see
Table 1.2 for details).2
In the following sections, we discuss the relationship between technology
acceptance and emotions, and explain the methodology applied in this research.
We then present our findings. Following an overview of technology-related
letters, we analyse the data from two more focused perspectives: we first look at
the themes of time and change; we then map the letters against Jacob Nielsen’s
(2012) widely applied usability matrix.
Emotions and technology acceptance

The starting point of this chapter is the well-known scientific fact that human
cognition is closely intertwined with emotions, and thus our reactions and opinions
towards any objects and matters in our lives are coloured by our emotions:
[F]ew if any perceptions of any object or event, actually present or recalled

from memory, are ever neutral in emotional terms. Through either innate
design or by learning, we react to most, perhaps all, objects with emotions,
however weak, and subsequent feelings, however feeble.
(Damasio 2004: 93)
More specifically, emotions also play a major role in acceptance of new

technology. In various technology-acceptance models widely used in fields linked
to IT and human-computer interaction (e.g., Zhang and Li 2005; Venkatesh and
Bala 2008), attitudes and emotions towards technology have been identified as
a central element of perceived ease of use, or the belief that using a particular
system will be fairly effortless. This perceived ease of use, in turn, together with
perceived usefulness, is fundamental to the user’s willingness to use a particular
tool (Venkatesh and Bala 2008).
Technology-acceptance models have not yet, to our knowledge, been employed
in translation studies, but effortlessness and usefulness are directly connected to
usability research, which has recently gained more recognition within translation
research (e.g. Byrne 2010; O’Brien et al. 2010; Suojanen et al. 2015). In a well-
known usability definition created by Jakob Nielsen (2012), the main attributes of
usability are listed as learnability, efficiency, memorability, errors and satisfaction.
We will return to these attributes shortly. Here, we would like to emphasize that
10 Kaisa Koskinen and Minna Ruokonen
both efficiency and satisfaction are linked to ease of use and effortlessness: an
efficient system increases the user’s productivity, because it is effortless and easy
to use, and this is also likely to give rise to a sense of satisfaction.
Thus emotions, attitudes and (perceived) usability all interact and influence
acceptance and adoption of new technology. Additionally, while emotions may
appear ‘soft’, technology acceptance carries significant economic weight: low
adoption and underutilization of IT systems have been identified as prominent
factors in explaining why IT investments may fail to increase productivity
(Venkatesh and Bala 2008: 274). It follows that insights into translators’ attitudes
and emotions, together with perceived ease or unease of use – in other words, their
user experiences – are also economically meaningful.
To summarize, translators’ attitudes towards technology and their experiences of
its usability are likely to influence the degree to which they adopt new technology.
It therefore seems necessary to chart the significance that translators attach to
technology in general and to translation-specific technology in particular. In this
chapter, we approach technology acceptance from two perspectives: first, that of
translators’ attitudes; and second, that of the relationship between user experience
and perceived usability, in which ease of use, as well as user satisfaction, play an
important role. In the following section, we justify and describe the method we
chose for this purpose.
Research methodology and the respondents

As pointed out, the method of writing a love letter/break-up letter to a tool or
device has been used in usability research in cases in which the focus is on the
emotions attached to devices and objects (Hanington and Martin 2012: 114). This
qualitative, free-form method was deemed preferable to predefined technology-
acceptance models for two reasons: first, because the method is designed to
trigger emotional responses, it is ideal for teasing out emotions and attitudes
towards various tools; and second, because the respondents were instructed
to freely choose the subject of their letter, the data will allow us to see how
extensively, if at all, translation-specific technology figures when compared to
other aspects of work.
Because the purpose of the present study was to find out the relative status and
role of technology in translators’ mappings of positive and negative emotions, the
instructions for the task were given in a very broad manner, and the respondents
were instructed to trust their first instincts and not to think too analytically before
responding. The instructions also focused explicitly on experience and emotions,
repeating words such as ‘pleasure’:
Picture yourself in the space where you normally work with your translation
assignments. Try to capture your first, intuitive reaction to the following
question: What is the greatest tool or support for you when you are
translating? What gives you the most pleasure? What would you be most
reluctant to lose?
Table 1.1 All letters by respondents
Love letter Ambivalent letter Break-up letter Total
EU translators 39 3 22 64
FI translators 20 3 14 37
MA students 24 3 20 47
Total 83 9 56 148
Or do some reverse thinking: What is the most annoying hindrance you

need to deal with? Which tool, artefact or element is emotionally the most
unpleasant? What would you be happy to get rid of?3
Participation was entirely voluntary in each group and the respondents were free
to write either a love letter or a break-up letter, or both. During spring 2014, a total
of 148 letters were collected from 102 respondents (see Table 1.1), comprising:
• 44 professional translators in the European Union (EU) institutions;

• 26 professional translators working on the Finnish market;
• 21 Master’s-level translation students in Finland; and
• 11 Master’s-level translation students in Ireland.
Data collection raises some issues related to the selection of participants and
to priming that may foreground technology. The EU data was collected in the
context of a training session dealing with the future of translation technology;
respondents were thus both, first, selected from among those interested in the topic
and, second, primed to think about technological tools, perhaps to the detriment
of other potentially more affective elements of their work. The Finnish data
was collected through an online survey administered via social media, pooling
respondents who follow digital media. Both sets of student data were collected
during a visiting lecture: one on localization and usability (with potential selection
towards technologically oriented students, but with the content more linked to
user experience and emotions than to translation technology); the other on
fieldwork methods (no obvious priming effects). (For a more detailed discussion,
see Koskinen 2014.) However, to avoid pushing the respondents too heavily in
one direction only, the respondents were explicitly encouraged to have a wide
perspective on supports and hindrances:
. . . You do not, however, need to focus on specific translation tools or

technology only. The most lovable element can also be your chair, your
view, or your coffee mug, and you may hate thin walls, the font used in the
documents, the hum of air-conditioning . . .
Because each subgroup is rather small, quotations from the material will not
be accompanied by any information about the respondents’ backgrounds, age
or gender, because such information may inadvertently result in exposing the
respondent’s identity. The respondents will be referred to by subgroup and code
number only, for example ‘FI-61’.
As a method, the love letter/break-up letter invites respondents to produce
narratives of emotion. It is important to stress that our data thus represents
reported affect and discursively constructed tales of emotion; it does not allow
direct access to the respondents’ psychological states. The data also contains
retrospective material as opposed to data collected in the context of actual
technology use (cf. Olohan 2011: 352–4). Collecting the latter kind of data
would ideally require an observational fieldwork approach, since laboratory
settings would not allow a realistic scenario. To our knowledge, such fieldwork
data focusing on translators’ affective and emotional responses to technology
use has not yet been collected; while LeBlanc’s interviews and observations may
offer some pointers on these themes, his focus is broader (LeBlanc 2013: 1–2).
Existing comparable research works with either survey data (Dillon and Fraser
2006: 68; Marshman and Bowker 2012) or Internet discussions (Olohan 2011) –
that is, similarly reported and narrative-performative data.
On the other hand, retrospective and holistic research material such as these
love and hate letters can be revealing in terms of long-term attitudinal factors as
opposed to fleeting affects. The respondents were allowed to freely choose the
object of their letter, as well as positive or negative focus, and they were also
asked to trust their first instinct and not to hold back. The data thus allows us to
assess how central technology was considered to be by the respondents. We can
also use the data to revisit the commonly held assumption that translators tend
to have ‘a negative mindset’ or ‘reluctance’ towards using technological tools
(Drugan 2013: 24). In interpreting the results, however, it needs to be noted that
the method introduces a forced binary model, pushing responses into one of the
two categories. This binarism is as much an asset as a hindrance in the analysis,
because the results are pushed towards clear-cut classifications.
An Excel spreadsheet was created to organize the entire set of 148 letters
for analysis, because Excel allows for easy cross-referencing against different
background variables (such as age, gender or subgroup). In our analysis, we
embrace the binary nature of the method, dividing data into two main categories
(love vs break-up letters). However, some nine letters were so ambivalent in tone
that enforcing the binary division would have been misleading. We therefore
created a third category for these nine letters. After this basic categorization, 106
letters containing technology references were filtered from the data. This subset of
data was then classified separately in terms of time and technological change (see
‘Time, experience and technological change’), and of usability (see ‘Usability and
user experience’). At all stages of the analysis, we applied a consensus method of
discussing and debating each item until we reached agreement on its classification.
The categories are based on several qualitative re-readings of the textual data.
We wish to emphasize that although we operate with numbers, the analysis is
predominantly qualitative in nature. It is a fairly common mistake to assume that
counting works only for a quantitative approach. The results that we present in
this chapter are strictly based on our joint qualitative interpretations of the data
and the data is also fully qualitative in nature. We use thematic categorizations
and counting of relevant items to tease out trends and to visualize our findings, but
the main thrust is essentially qualitative.
(Translation) Technology as an object of love and hate
Overview
In contemporary discussions, translation technology is often reduced to
translation memory (TM) tools only. Of course, TM tools are a central feature
of the contemporary translation industry, which means that it is indeed relevant
to study and discuss their effects and usability. However, an overemphasis on
TM can easily obscure the full extent to which translation is technology-driven.
Translators who responded to the call for letters adopted a much wider perspective
in reporting on technological tools that either help or hinder them in significant
ways in the course of their daily work. For the purposes of this study, the category
‘technology’ was accordingly defined rather broadly: in addition to prototypical
translation technology such as TM software and machine translation (MT)
systems, we included other software (such as word processing, time management
systems and operating systems), search tools and databases (the Internet, Google,
IATE, etc.), hardware (laptop, mouse, keyboard) and references to ‘computers’ or
‘IT’ in general (cf. Fulford and Granell-Zahra 2005).
Based on this definition, technology was by far the most discussed theme in all
letters: of the 148 letters, a total of 106, or some 70 per cent, engaged directly with
some aspect of technology. Because some respondents chose to write both love
and break-up letters, the total number of respondents who addressed technology
was 79, or 78 per cent of the respondents. This suggests that the respondents
consider technology to be central to translators’ work. Table 1.2 further shows
how many respondents in each group wrote love, break-up or ambivalent letters
in which technology was at least mentioned.
While most love letters included positive comments on technology, there
were three love letters in which technology was discussed in a negative tone.
Conversely, one break-up letter made positive comments on technology. Even
when this is taken into account, however, it is clear that over half of the letters
(57) manifest a positive attitude towards technology, whereas only 40 discuss
technology in a negative sense. This indicates that translators are hardly as averse
to technology as has sometimes been suggested.
Interestingly, practising translators, particularly those working in the EU,
wrote love letters to technology more frequently than did the translation students,
who are rather divided in their attitudes. This may perhaps be explained by the
fact that translators have more work experience and thus more experience in using
technology – a point to which we return later.
We next examined the objects of love, break-up and ambivalent letters more
closely to discover which technological phenomena occurred in the data and for
what reasons.
Table 1.2 Technology letters by category and respondent
EU translators 27 1 13 41
FI translators 16 3 12 31
MA students 16 3 15 34
Total 59 7 40 106
Table 1.3 Letters commenting on translation-specific tools
Positive tone 14 1 0 15
Ambivalent tone 4 3 0 7
Negative tone 0 0 13 13
Total 18 4 13 35
Of the love letters to technology, the largest subgroup was the 22 letters
dedicated to various search tools and databases. Another large subgroup was the
10 love letters addressed to TM software. Both search tools/databases and TM
software were also mentioned in several other letters in a positive tone. Computers
in general received nine love letters; there were also letters written to hardware
such as keyboards, screens or an ergonomic mouse.
While the scope of artefacts that received loving mentions was perhaps
unexpectedly wide, it is still clear that search tools were by far the most
widely appreciated tool. Their role is even further emphasized when we take
into consideration that, in addition to the technologies mentioned above, there
were seven love letters to traditional printed dictionaries, and four to research
as such and the joy of discovering accurate equivalents or useful parallel texts.
As described by one of the respondents, ‘searching books and the Internet for
information is the best part of translation’ (FI-61, authors’ translation).
The break-up letters are, in many ways, the mirror image of love letters.
The object of one’s love easily turns into an object of hate if it fails to function
as expected. While ‘the Internet’ and ‘computers’ in general received many
declarations of love, break-up letters were more specifically addressed to a ‘slow’
or ‘erratic’ Internet connection or to the translator’s computer ‘when it’s acting
up’. On the whole, technology is at least one object in 40 break-up letters of the
total of 56 and the aspects covered can often be linked to usability: technological
tools are criticized for being slow, unreliable or difficult to use. These and other
aspects of usability are analysed more closely later in the chapter.
We further analysed whether translation-specific tools (translation memories,
machine translation, terminology tools, translation management systems) were
mentioned in a positive or negative tone or both. The results are shown in Table 1.3.
Letters mentioning translation-specific technology thus constitute 31 per
cent of the technology-related love letters (18 out of 59) and 33 per cent of the
technology-related break-up letters (13 out of 40). Considering that all translators
who responded to the survey most likely make use of TM software in their
work,4 the percentages are not that high. For example, in Matthieu LeBlanc’s
interviews, most translators at three Canadian translation agencies seem to have
commented on translation technology without prompting (LeBlanc 2013: 6). The
relatively few references to TM software in our data may indicate that translation
memories have become such a regular part of translators’ work that they have
become invisible to a certain degree. Indeed, only a few TM programs were
mentioned by name in the letters: (SDL) Trados Studio, which is the house tool
for EU translators and probably remains the most widely used TM software,5
came up in four love letters and six break-up letters, while MemoQ received two
love letters and Wordfast, one. The most loved individual translation tool in this
data was, however, not a TM tool, but Quest (six letters). Quest is a metasearch
tool developed by the EU’s Directorate-General for Translation (DGT) to
enable terminology searches from several EU internal and public databases
simultaneously, designed to drastically reduce time spent on searching for correct
terms (DGT 2012: 10). None of the letters mentioned translation management
systems, whether in positive or negative terms. This may suggest that while
controlled document management is important for successful professional
practice, it is not seen as a central aspect of translation and also is not necessarily
the responsibility of translators in all work contexts. We may also conclude that
the respondents were perhaps generally rather satisfied with the systems at their
disposal, as these did not come up as targets of hate mail.
Machine translation (MT) figures in the letters less than TM, and there is
a clear division in terms of which respondents comment on it and how. First,
while most Finnish business translators apparently rarely use MT tools (Wivolin
and Niskanen 2012), all EU translators had recently migrated to a system that
integrates MT and TM. This probably explains why MT does not figure in the
subgroup of Finnish translators at all; in contrast, six EU translators wrote love
letters to their institutional TM software, which incorporates MT, and only one
EU translator wrote a break-up letter railing against ‘cumbersome, tyrannic MT
systems’ (EU-44). Second, all of the letters addressed to Google Translate, which
is the best-known free MT system, were written by MA students, who either
expressed concerns about its effect on their professional future or doubts about
the quality of the translations it produces. Of the letters to Google Translate, three
were break-up letters and only one was a love letter (and the author of that letter
also wrote a break-up letter to the software in question). The absence of letters by
professional translators may suggest that they do not consider Google Translate
to be serious competition (cf. Katan 2009: 130, 132), that it does not figure as a
tool in their work, or both.
On the whole, the letters do not lend themselves to drawing conclusions about the
role of translation-specific tools. In terms of acceptance of new technology, these
results are inconclusive, but, as Table 1.3 shows, positive and negative comments
are fairly evenly divided, and the positive comments even slightly outnumber
the negative ones. Translators can thus hardly be seen as ‘anti-technology’, as
Table 1.4 Translators’ vs students’ comments on translation-specific technology
EU/FI translators MA students
Positive tone 14 1
Ambivalent tone 4 3
Negative tone 9 4
Total 27 8
Table 1.5 Love and break-up letters and their attitudes towards change
Past Future
Love letter Break-up letter Total Love letter Break-up letter Total
Positive tone 2 0 2 1 3 4
Negative tone 8 1 9 0 1 1
sometimes claimed. Of the respondents who commented on translation-specific

technology in particular, the majority were practising translators rather than
students, as shown in Table 1.4. This is probably explained by the translators’
broader exposure to translation technology in the course of their daily work.
The practising translators’ more extensive experience of translation technology
may also explain why they made so many more positive comments than the student
respondents: they have had the time to hone their skills and to accumulate positive
experiences. To dig deeper into potential differences between translator generations,
we look more directly at attitudes to technological change in the next section.
Time, experience and technological change

When investigating translators’ acceptance of new technologies and attitudes
towards technological progress, it is relevant to analyse how they describe
their contemporary working life as compared to either the past or the future: do
translators feel that everything was much better before and do they perceive the
future as threatening or full of promise?
The notion of time was a factor in 23 letters concerning technology (16
love letters, 5 break-up letters and 2 ambivalent letters). Table 1.5 presents the
frequencies of those love and break-up letters covering technology that included a
temporal comparison and characterized the change as either positive or negative.
Most of the letters with a temporal dimension were written by practising translators.
Only two of these letters were written by students: one saw the past as worse than
the present; the other believed that the future would be worse than the present. This
makes sense, because students do not have much experience of previously used
tools, but it also indicates that they do not focus on a forward-looking approach.
With a total of nine negative comments on past technologies, Table 1.5
shows a tendency to consider past conditions inferior to the present, suggesting
that technological progress was appreciated rather than deplored. There is a
Table 1.6 Work experience vs positive and negative affects towards technology
Work experience Respondents Love Ambivalent Break-up

(n) letter letter letter
Junior translator = < 10 years 20 (10 EU + 10 FI) 14 1 5

Senior translator = > 20 years 21 (14 EU + 7 FI) 12 1 11
MA student 32 15 3 16
much slighter tendency to expect the future to be brighter than the present (four
comments), indicating a more hesitant, but still mildly optimistic, approach to
technological change in the future. The remaining time-related letters that we felt
were not classifiable into these categories do not alter the picture: they are either
neutral in tone, depict the ups and downs of an evolving relationship between the
translator and the tool, or stay positive across time.
In two letters, the issue of time and change was explicitly addressed. One
respondent claimed that ‘ “technology”-wise EU institutions lag so much behind
the industry’ (EU-15). Another suspected that:
. . . not all colleagues share my enthousiasm [sic] [for search tools], usually
senior translators who have acquired their knowledge without technology &
tools. Their disgust of or disrespect for such tools puzzles me—. But we have
the generation gap to thank for that I guess—
(EU-34)
These two letters seem to offer some support for the findings of Sarah Dillon and
Janet Fraser’s (2006: 73–5) survey in which younger translators were found to
have a more positive attitude towards TM tools than more senior translators. This
induced us to explore whether our data shows differences between respondents
with varying amounts of work experience and whether the respondents with
less experience would feel more positively about translation technology. To
do this, we compared the 20 respondents with less than 10 years’ experience
to those 21 in our data who had been professional translators for more than 20
years to see whether they had opted for a love letter or a break-up letter related
to technology. Both groups were also compared to MA students. The results are
shown in Table 1.6. The figures have been corrected to match the content of the
letter, so that love letters (to something else) with a negative view of technology
are classified as break-up letters and break-up letters with a positive view of
technology are counted as love letters.
In groups of both junior and senior translators, love letters outnumber break-up
letters, although only by one letter in the senior group. Since previous research
seems to suggest a general and possibly increasingly negative mindset (as
discussed earlier and in the introduction), we take this balance as an indicator
of senior translators’ unexpectedly positive affects towards technology. One
respondent beautifully encapsulates a long and ultimately positive relationship:
Dear Internet,
This is the first time I’m writing to you although my feelings for you have been
strong and warm for a long time. I remember when we were both still young and
foolish, but since the bubbly infatuation of our first encounters my feelings have
gradually deepened as we have both matured and grown to know each other
better, and I could no longer imagine a single working day without you.—
I know, my dearest Internet, that some people find you unreliable, but I have
learned to take you as you are and act accordingly – after all, there are always
two parties to a relationship, and I couldn’t require you to be absolutely
reliable but must be aware of my responsibility and duty to make an effort
and be critical of the sources you offer. It is in this way that we can both keep
our relationship rewarding.
Love,
(FI-48, authors’ translation)
In the junior cohort, break-up letters are much fewer in number: 5 break-up letters
as opposed to 14 love letters. The difference is partially explained by the seniors’
greater willingness to produce two letters rather than just one: out of the 21 senior
respondents, 9 wrote both a love and a break-up letter to technology, while out of
the 20 junior respondents, only 5 produced two letters. Thus it could be argued that
junior translators’ overall attitudes towards technology appear more optimistic,
while the senior translators evince more mixed, or even critical, feelings. On
the whole, however, it can hardly be argued that there is a clear generation gap
between junior and senior translators.
The next logical step is to consider whether a possible ‘generation gap’ can
be detected among translation students, who were the most junior cohort in this
study, nearly ready to enter the profession – although one third of the students
already had some translation experience. They were also the youngest in age: only
six professional translators reported their age to be between 21 and 30, whereas all
but five students were aged 30 or younger. Previous research (Dillon and Fraser
2006: 68; Marshman and Bowker 2012: 76) suggests that this cohort would be the
keenest to engage with modern technology. However, in our data, this does not
seem to be the case. Remarkably, the student group is the only one among these
cohorts in which break-up letters outnumber love letters, which therefore makes
them the cohort expressing the most disaffection towards technology.
The students’ reluctance may partially reflect the students’ precarious
position compared to those who already have found their professional foothold:
in the entire sample of letters, students stand out as the group that expresses
the most worry about the profession and its future, or their own future as part
of it (Koskinen 2014: 82). In this respect, the results also raise questions as to
how positively future workplaces are described at training institutions and how
translation trainers approach technology. Surveys comparing translation trainers’
and students’ attitudes to technology have been conducted in connection with
the Collection of Electronic Resources in Translation Technologies (CERTT)
project at the University of Ottawa. In these surveys, the students reported
higher comfort levels with technologies in general than the trainers (Marshman
and Bowker 2012: 77), but, when asked about barriers to using translation
technology in particular, over 15 per cent of the students reported discomfort
as an obstacle, whereas none of the trainers mentioned this (Marshman and
Bowker 2012: 79). Further investigation of both translation students’ and
teachers’ attitudes towards technology would clearly be of interest.
Usability and user experience

While love letters and break-up letters to various technical devices can be
interpreted as signals of attitudes and affects, they also need to be taken at face
value, as indications of usability. As heavy users of various tools and applications,
translators are well placed to appreciate well-functioning ones and to exhibit
negative emotions towards those that interfere with their work. Furthermore,
translators are seldom actively involved in the design or test phases of their tools
(O’Brien et al. 2010). It is thus not unlikely that translators’ user experiences
may, in some cases, be even severely compromised.
Because respondents were solicited to write a letter to the most pleasurable or
most annoying tool or feature of their work, the data allows us to observe some of
the most memorable user experiences, positive and negative, of an occupational
group that has been presented, during the past decades, with a growing number
of technological devices radically affecting their work. The most senior cohort,
in particular, has seen and experienced the birth of such fundamental features of
contemporary translation as the Internet, online databases and TM; the most senior
respondents may have even witnessed the advent of word processing. On the whole,
the pool of experiences provided by 102 professionals and students is a rich source
of information on translators’ opinions concerning the usability of technology.
As the basis of our analysis we chose a well-known usability definition created
by usability consultant Jakob Nielsen. According to Nielsen (2012), the main
components of usability are:
1 Learnability: How easy is it for users to accomplish basic tasks the first time
they encounter the design?
2 Efficiency: Once users have learned the design, how quickly can they
perform tasks?
3 Memorability: When users return to the design after a period of not using it,
how easily can they re-establish proficiency?
4 Errors: How many errors do users make, how severe are these errors, and
how easily can they recover from the errors?
5 Satisfaction: How pleasant is it to use the design?
We mapped all technology letters against these five components to see which ones,
if any, would stand out as most relevant for translators. The element of satisfaction
is in-built in the love/break-up letter method, and all letters, to some extent, thus
20
24
3 3
3
Learnability System errors

Efficiency Production errors
Memorability Satisfaction
Figure 1.1 Usability and positive affects
belong to this category and express sentiments of (dis)satisfaction. In this analysis,

we break down the data in more detail to tease out different aspects of usability.
The attribute of satisfaction is thus delimited to direct verbalizations of emotional
or physical (dis)comfort, such as ‘I enjoy technologies’ (EU-15), ‘I’ve been able to
heal my aching arm’ (FI-53, authors’ translation), or ‘You make my life so much
easier and my work so much more fun and enjoyable’ (MA-74). It may still be that
the category of user satisfaction is overrepresented in our data because the method
invites the respondents to foreground experiences and emotions.
During the analysis, we also realized that we needed to modify one of Nielsen’s
components to suit our particular case. Nielsen’s fourth component focuses on
errors that take place in using the system, whereas our data also contains a number
of comments on the errors that the system may introduce into the end product –
that is, the translation. We thus split the category of errors into system errors and
product errors in our analysis.
Figures 1.1 and 1.2 show how frequently the five attributes on Nielsen’s list
were commented on in the letters. Not all technology letters contained references
that can be linked to usability; of those that did, many letters contained elements
of more than one attribute. None of the break-up letters discussed aspects of
usability in a positive light, but there were 10 instances of negative comments in
love letters, and seven positive and six negative comments in ambivalent letters.
Figures 1.1 and 1.2 incorporate results for all three types of letter.
It is easy to discern that positive affects are dominantly linked to either efficiency
or user satisfaction; in 10 letters, both were mentioned. Several respondents wrote
in their love letters that a search tool such as Google speeds up their work, ‘offering
4 4
10
17
27 4
Learnability System errors

Efficiency Production errors
Memorability Satisfaction
Figure 1.2 Usability and negative affects
answers in the blink of an eye’ (MA-84, authors’ translation), or that, without it,
‘every translation assignment would take days to complete’ (EU-1). Satisfaction
was often linked to tools that make the translator’s work easier, but some tools
were simply enjoyable or even fun to work with: ‘[Without the Internet,] my life
as a translator would definitely be more tedious and more tiring’ (EU-23).
Since efficiency came out as a significant positive element in the usability of
translation tools, it is not a surprise to see lack of efficiency considered a major
cause for a desire to ‘break up’. Typical letters to inefficient tools criticized having
to wait for the computer or software to start up, or being obliged to click several
times to perform simple operations.
Interestingly, by far the greatest number of negative comments concern errors –
in particular, system errors (i.e. functional problems in tools or programs). This can
lead to high levels of frustration. Lack of errors – that is, error-free systems – did
not receive many mentions, indicating a thinking pattern familiar to translators: a
beautifully running system is taken for granted; the system is noticed only when
there is an error.
All in all, the attributes of usability to receive the most comments in the letters
were efficiency (41 comments), satisfaction (30 comments) and system errors (30
comments); the other attributes received 7 comments or fewer. Efficiency thus
appears central to translators’ experience of usability.
We have already mentioned that perceived ease of use is considered a
fundamental element of technology acceptance. Our data shows that our
respondents indeed placed a high premium on such attributes of usability as
efficiency and user satisfaction, assigning significant positive value to them.
Similarly, lack of efficiency, and system errors that can be considered an extreme
case of inefficiency, rank highest among negatively valued usability factors.
In contrast, learnability and memorability do not seem to be an issue, either
positively or negatively. On the one hand, one could perhaps argue that translators
are a tech-savvy professional group who are used to using different tools and
migrating from one to another, and that they do not have problems with learning
new ones. On the other hand, it is also possible that translators do not wish to
appear clumsy with technology and so they disguise their potential insecurities as
criticisms linked to other attributes of usability. (Of course, in usability thinking,
difficulty of learning is also decisively an attribute of the system, not of the user.)
Discussion and conclusions

In this chapter, we set out to explore translators’ emotional narratives about
their work to discover how centrally technology figures in translation practice,
what kinds of emotions translators express towards technology and how they
experience its usability. For this purpose, the love and break-up letters – 148 in
all – proved rich and illuminating material.
The letters demonstrate that technology is indeed a central and, for the most
part, positive aspect of translators’ work. In addition to translation-specific tools,
translators regularly use and appreciate many general-purpose systems and
applications, such as the Internet. Various forms of technology were by far the
most discussed theme in the letters in general, with some 70 per cent of the letters
addressing some form of technology. Furthermore, over half of the letters expressed
positive attitudes toward technology and those letters that discussed technology in
terms of time mainly evinced a sense of improvement. There was no evidence
of a generation gap between junior and senior translators; in fact, the group that
seems to feel the most sceptical about translation technology is that of translation
students, probably because of their lack of practical experience. Thus the results
of this study do not support the notion of translators being averse to technology.
Although translators do not dislike technology, however, they do dislike non-
functioning technology and poor usability. When analysed on the basis of the
attributes of usability, the letters highlight, first, the importance of efficiency and
improved productivity. As mentioned in the introduction, efficiency or speed is
also a major rationale for introducing new translation tools and the fact that it
spontaneously comes up in the letters so frequently is in line with the fact that
efficiency is an important aspect of contemporary translators’ work. This further
suggests that translators are probably quite willing to adopt new technology as
long as it makes their work more efficient. Our data also indicates that usability
problems are a significant barrier to efficiency and productivity in the translation
industry, and give reason to argue that increased user involvement in the design
and development of translation-specific tools could lead to added ease of use.
For example, the metasearch tool Quest that the DGT itself developed ranked the
highest in evoking unsolicited positive emotions in our data.
The findings reported in this chapter are the result of an exploratory project with
the aim of testing the love/break-up letter method. The method has proven both
unexpectedly fruitful and also well liked by those participating in this research.
It can definitely be recommended alongside other methods aimed at soliciting
translators’ attitudes and emotional attachments. At the same time, it is clear that a
free-form narrative method can provide only indications or trends, not hard data on
actual attitudes. The results indicate further avenues of research by other methods.
Technology-acceptance models often rely on empirical observations and useful
information on the severity of usability problems, for example, could be obtained by
shadowing translators at their work. Translators’ technology attitudes and perceived
ease of use can also be mapped through more quantitatively oriented survey
methods or using interviews. Technology, translation-specific or not, will remain a
central shaping force in translators’ work for the foreseeable future. The better we
understand all of the factors that affect translators’ willingness and ability to adapt
› changing technology, the better we understand professional translation practice.
Notes
1 The emotional side of translation has, in general, largely remained outside the scope
of research, although there are some early contributions, such as Robinson (1991) or
Hansen (2005). In recent years, researchers have increasingly embraced issues of affect,
in terms of both how to convey the affects expressed in the source text (e.g. Shields and
Clarke 2011) and how translators’ own affective responses constrain and/or support their
professional performance (e.g. Lehr 2014).
2 For an analysis of all letters and for a more detailed account on data collection and group
characteristics, see Koskinen (2014).
3 This quotation is from the English-language template. The Finnish participants were
given similar instructions in Finnish.
4 All EU translators have access to TM tools and, in a 2011 survey by the Finnish
Association of Translators and Interpreters, the vast majority (over 70 per cent) of
the responding Finnish professional business translators used TM tools in their work
(Wivolin and Niskanen 2012).
5 In Lagoudaki’s (2006: slide 26) survey, with almost 900 respondents from more than 50
countries, TRADOS was used by 51 per cent of the respondents, SDL Trados 2006 by 24
per cent and SDLX 19 per cent (the respondents could choose multiple programs). Since
2009, MemoQ in particular has gained popularity, but SDL Language Technologies still
claims to be the ‘world leader’, with 200,000 licences (http://www.translationzone.com/
about).
References
Byrne, J. 2010. Technical Translation: Usability Strategies for Translating Technical
Documentation. Dordrecht: Springer.
Damasio, A. 2004. Looking for Spinoza: Joy, Sorrow and the Feeling Brain. London:
Vintage Books.
Dillon, S., and J. Fraser. 2006. ‘Translators and TM: An Investigation of Translators’
Perceptions of Translation Memory Adoption’. Machine Translation 20(2): 67–79.
Directorate-General for Translation (DGT). 2012. Translation Tools and Workflow.
Brussels: European Commission. Available at: http://bookshop.europa.eu/is-bin/
INTERSHOP.enfinity/WFS/EU-Bookshop-Site/en_GB/-/EUR/ViewPublication-
Start?PublicationKey=HC3212080 [accessed 12 March 2016].
Drugan, J. 2013. Quality in Professional Translation: Assessment and Improvement.
London: Bloomsbury.
Fulford, H., and J. Granell-Zahra. 2005. ‘Translation and Technology: A Study of UK
Freelance Translators’. Journal of Specialised Translation 4: 2–17.
Hanington, B., and B. Martin. 2012. Universal Methods of Design: 100 Ways to Research
Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Beverly,
MA: Rockport.
Hansen, G. 2005. ‘Experience and Emotion in Empirical Translation Research with Think-
Aloud and Retrospection’. Meta 50(2): 511–21.
Katan, D. 2009. ‘Translation Theory and Professional Practice: A Global Survey of the
Great Divide’. Hermes 42(7): 111–53.
Koskinen, K. 2014. ‘Kääntäjän habitus fiktiivisten rakkaus-ja erokirjeiden valossa’.
MikaEL 8: 74–88.
Lagoudaki, E. 2006. ‘ICL Translation Memories Survey 2006’. Presentation at Translating
and the Computer 28, London, 15–16 November. Available at: http://www.slideshare.
net/elinalag/icl-translation-memories-survey-2006 [accessed 12 March 2016].
LeBlanc, M. 2013. ‘Translators on Translation Memory (TM): Results of an Ethnographic
Study in Three Translation Services and Agencies’. Translation & Interpreting 5(2): 1–13.
Lehr, C. 2014. The Influence of Emotion on Language Performance: Study of a Neglected
Determinant of Decision-Making in Professional Translators. PhD thesis. University
of Geneva. Available at: https://archive-ouverte.unige.ch/unige:42306 [accessed 12
March 2016].
Marshman, E., and L. Bowker. 2012. ‘Translation Technologies as Seen through the Eyes
of Educators and Students: Harmonizing Views with the Help of a Centralized Teaching
and Learning Resource’. In M. Borodo and S. Hubscher-Davidson (eds). Global Trends
in Translator and Interpreter Training: Mediation and Culture. London/New York:
Continuum, 69–95.
Nielsen, J. 2012. ‘Usability 101: Introduction to Usability’. 4 January. Available at: http://www.
nngroup.com/articles/usability-101-introduction-to-usability/ [accessed 12 March 2016].
O’Brien, S. 2012. ‘Translation as Human–Computer Interaction’. Translation Spaces 1(1):
101–22.
O’Brien, S., M. O’Hagan and M. Flanagan. 2010. ‘Keeping an Eye on the UI Design
of Translation Memory: How do Translators Use the “Concordance” Feature?’. Paper
presented at the European Conference on Cognitive Ergonomics, Delft, 25–28 August.
Available at: http://doras.dcu.ie/16693/ [accessed 12 March 2016].
Robinson, D. 1991. The Translator’s Turn. Baltimore, MD/London: Johns Hopkins
University Press.
Shields, K., and M. Clarke (eds). 2011. Translating Emotion: Studies in Transformation
and Renewal between Languages. Frankfurt: Peter Lang.
Suojanen, T., K. Koskinen and T. Tuominen. 2015. User-Centered Translation: Translation
Practices Explained. London: Routledge.
Venkatesh, V., and H. Bala. 2008. ‘Technology Acceptance Model 3 and a Research
Agenda for Interventions’. Decision Sciences 39(2): 273–312.
Wivolin, S., and E. Niskanen. 2012. ‘II jaoston taustatietokysely: loppuraportti’. Available (to
members only) at: https://sktl-fi.directo.fi/jasenet/asiatekstinkaantajat/arkisto/arkistoidut-
tapahtumat-ja-uutise/2012/7-2-2012-kyselyn-esittely/ [accessed 12 March 2016].
Zhang, P., and N. Li. 2005. ‘The Importance of Affective Quality’. Communications of the
ACM 48(9): 105–8.
2 Deconstructing translation
crowdsourcing with the case
of a Facebook initiative
A translation network of engineered
autonomy and trust?
Minako O’Hagan
Introduction
The technologization of translation, a process that has been under way for some
time, took a new turn with the advent of the Internet. The creation of a digitally
connected world has contributed to an increased and more varied demand for
translations, and also a wider range of solutions, including online machine
translation (MT) and multilingual resources, made accessible to translation
users and providers alike via ever-expanding networks. More recently, the
second-generation Web technologies commonly known as ‘Web 2.0’ have
turned the Internet into a locus of distributed problem solving that taps into
human intelligence through online participation. Described as ‘a connective and
collaborative technological environment that enables individuals to get involved
in internet-mediated social participation, communication, and collaboration’
(Zhao and Zhu 2014: 418), the second-generation Internet has become a highly
connected and interactive place that forms people into networks. In particular,
Web 2.0 has focused attention on general Internet users who contribute user-
generated content (UGC) that is distributed and shared among the global online
population (Howe 2006). It is in this context that translation itself has emerged
as user-generated (O’Hagan 2009; Perrino 2009), with translation being carried
out in a collaborative network, as in the case of ‘fan translation’, whereby
dedicated fans translate their favourite foreign-language content (O’Hagan 2009).
In contrast to this popular, yet largely illegal, form of UGC is the relatively
recent phenomenon of the purportedly legitimate solicitation of labour through
‘translation crowdsourcing’. Web 2.0 applications provide a mechanism for
the formation of an ad hoc translation workforce through open calls from non-
profit and for-profit organizers of various translation initiatives. Often seen as
the catalyst for ‘the rise of the amateurs’ (Howe 2006), the Internet opened up
translation as an everyday online activity performed by self-declared translators
who produce translations in response to open requests. Facebook’s translation
campaign has most comprehensively demonstrated the crowdsourcing model in
translation, whereby Facebook user communities were asked to help to translate
its website. Unsurprisingly, this caused outrage among professional translators,
26 Minako O’Hagan
who perceived it as nothing but a cost-cutting measure. Despite the controversy
and protests, however, the Facebook initiative, officially launched in 2008,
succeeded in making the originally English-only Facebook website available
in 75 languages within two years (Drugan 2013: 174), rising to 104 languages
and dialects by late 2013 (Dombek 2014: 4). Furthermore, the custom-designed
translation platform illustrates the key role in translation crowdsourcing played
by technology, especially tailored with social networking in mind (Wong
et al. 2014). Consequently, this phenomenon has attracted research interest in
translation studies focused on key concepts such as ‘collaborative translation’
(Désilets 2007), ‘community translation’ (O’Hagan 2011) and ‘non-professional
translation’ (Susam-Saraeva and Pérez-González 2012), among others.
Since then, a range of crowdsourcing-inspired practices have emerged among
both new and well-established language service providers (LSPs) (Drugan 2013;
Garcia 2015). Translation crowdsourcing can arguably be seen as one of the latest
manifestations of the deepening relationship between translation and technology
(Gaspari 2014), while the controversy surrounding this new model reflects the
dynamic nature of contemporary business, including translation, as it is reshaped
by ‘disruptive’ technologies and innovation (Christensen 2000). In this context,
it becomes increasingly important to gain an understanding of the impact of
technology on translation and translators to shed light on the human ‘cost’ of
technology – a key theme addressed in the present volume.
This chapter thus seeks to explore the evolving relationship between translation
and technology, focusing on the resultant human issues for translators. From a
sociological perspective, this study examines the human consequences of the
ongoing technologization of translation as manifested in translation crowdsourcing,
in which participants voluntarily engage in technology-mediated translation
networks. The study is designed to answer the central question: what does
translation crowdsourcing reveal about the role of technology in its interaction
with self-selected translators? In particular, this question is framed with reference
to translator autonomy and trust, both specific issues highlighted in the literature.
Both of these factors are an important part of translator agency and form a key
concern in sociological perspectives in translation studies (Abdallah 2012). Taking
the case of Facebook Translation crowdsourcing (FTC), this chapter examines how
technology provides a (dis)service to translators in terms of these two aspects in the
newly evolved practice described as ‘community translation on a social network’
(Wong et al. 2014). The analysis is conducted within the framework of actor–
network theory (ANT) (Latour 1987, 2004) to investigate how participants in an
FTC network form a heterogeneous group of ‘actors’ that interact and function in
fulfilling their goals, and in turn shape the network as a whole. According to ANT,
such actors can be human or non-human technological entities.
The next section focuses on professional translation today to contextualize
some of the human factors related to translator agency – that is, autonomy and
trust – arising from technologization. This is followed by a brief explanation
of ANT, which is then applied to FTC. The results are elaborated upon in the
discussion section, before the final conclusions are presented.
Crowdsourcing and the Facebook initiative 27
The role of technology in professional translation:
the issue of translator autonomy and trust
The technologization of translation is proving to be a more multifaceted
phenomenon than may have been predictable when the first experiments in
MT were conducted in the 1950s. Rather than being a matter of straightforward
automation using MT, technologization has come to involve the use of a
whole host of computer-assisted translation (CAT) tools and recyclable
linguistic resources. Added to this picture are the translation services of ‘cloud
marketplaces’ – that is, paid crowdsourced translation services offered by a new
breed of translation provider that are generally able to deliver the service at a
lower cost than conventional LSPs (Garcia 2015). For example, companies such
as Tokyo-based Gengo and Smartling, headquartered in New York, focus purely
on crowdsourced translation services, with varying approaches and systems for
the remuneration of their translators (Garcia 2015). Meanwhile Lionbridge’s
Enterprise Crowdsourcing,1 launched in February 2013, is an example of a well-
established major LSP adding a crowdsourcing model to its offerings. It includes
‘user-generated content translation’ based on ‘a private crowd – qualified and
managed by Lionbridge’.
The advent of Web 2.0 has affected many professional areas of work,
challenging the conventional model reliant on paid professionals (Howe 2006).
As a result, the ensuing debates about crowdsourcing include some extremely
negative views whereby non-professionals are seen as invading professional
domains of work, with a commensurate over-appreciation of amateurs, dubbed
‘the cult of the amateur’ (Keen 2007). Similarly, in the field of translation, the
group Translators for Ethical Business Practices2 was formed in 2009 to make the
case that translation crowdsourcing by Facebook and Twitter was unethical. Some
professional translators saw crowdsourcing as a mechanism for potential clients
to solicit translations cheaply, or even free of charge, from online volunteers
whose credentials as translators were considered dubious at best, undercutting
professional translators (Kelly 2009). Similarly, professional bodies such as
the American Translators Association (ATA) warned that such a solution was
detrimental to the clients’ own interest, given the likely quality problem (ATA
2009). In the meantime, a large-scale international survey in 2008 elicited some
900 responses from professional translators and interpreters (Katan 2011),
indicating that 65 per cent of the translators felt that their biggest competition was
‘amateurs’, including subject specialists who are not translators by profession.
In comparison, technologies (presented under the category of ‘e-tools’) were
perceived only as a ‘mild threat’ (Katan 2011: 73).
Katan’s (2011) study approaches the question of translation as an occupation
or a profession based on the analysis of the practitioners’ self-perceptions. He
presents some evidence that translation is a lower autonomy profession (LAP),
with ‘autonomy’ being defined as the ability to control new entrants into profession
and their subsequent practices, and as the exercising of ‘autonomous thought and
judgement’ and ‘responsibility to clients and wider society’ (Lester 2009: 2, cited in
Katan 2011: 73). In Katan’s study, translators express their feelings of helplessness
28 Minako O’Hagan
in being unable to control new entrants, while their attention is often locally focused
on the texts they translate rather than broader professional issues. This leads him to
suggest that translation signals survival at the individual level, rather than for the
profession as a whole (Katan 2011: 73). Following a similar line of argument, an
observation of new entrants in a paid crowdsourcing model has led Garcia (2015: 38)
to suggest that translation is becoming even more of a ‘fuzzy profession’, providing
further confirmation that ‘no one needs to be a professional translator to translate’
(Garcia 2015: 31). Katan (2011: 78) also notes, in his study, that Internet search results
with the keyword ‘translator’ collocate with such terms as ‘automatic’, ‘machine’ and
‘free-online’, thus representing translation as a ‘non-human, technical LAP’.
Ironically, now that translators no longer consider technology to be as much of
a threat as they did, the external image of translation and translators as reflected in
search engine results is that of an occupation that is overshadowed by technology.
This goes hand in hand with a lack of autonomy, as seen in the generally passive role
played by translators in the development and deployment of translation technologies
that are profoundly impacting on the work they do. The research literature on
translation memory (TM) is replete with examples of how translators constantly
adapt themselves to the technology rather than the other way round, with researchers
reporting variously what can be considered as evidence of technology limiting
translator autonomy. For example, novice translators show signs of ‘blind faith’
in making unquestioned use of TM matches (Bowker 2005). While TM-imposed
segmentation has been found to be counter-intuitive to the natural human translation
process (Dragsted 2004), thereby affecting translators’ cognitive load (O’Brien 2006),
the idea of sentence-based translation has become part of the processing pattern of
translators who regularly work with TM. In the meantime, TM has been widely
applied to all kinds of texts that are not necessarily repetitive in nature, even though
the technology was originally designed to be used for repetitive texts. Consequently,
there is a concern that overreliance on TM could lead to the deskilling of translators,
with negative implications for the development of translation competence (Kenny
2011; Drugan 2013). Such evidence further substantiates translators’ own perception
of translator autonomy being steadily undermined in today’s professional settings.
According to the literature on translation technology, technologized and
networked translation environments raise the question of trust (Abdallah 2012).
A sense of trust is an inherent human need and forms an important anchor in
the technology-driven translation ecosystem, in which translators working in the
context of TM/MT environments need to ‘learn to trust and mistrust data’ (Pym
2013: 495). As well as the need for an efficient retrieval mechanism, successful
translation recycling via TM ultimately relies on the quality of the translations stored
in the memory that are to be reused. This issue is particularly relevant in the face of
increasingly open and shared linguistic and translation resources; memory content
may be derived not only from quality-checked human translation, but also from
MT and post-edited MT, as well as crowdsourced translations. In the context of a
team-based translation workflow, Karamanis and colleagues (2011: 40–1) highlight
trust as a significant factor, finding that in-house translators’ input to TM was more
trusted than those of external translators. In such environments, trust has become
an important factor. In some CAT tools, ‘trust’ is flagged to the user explicitly in
terms of ‘confidence scores’. Both TM and MT, as well as online terminological
databases, use such scores to signal to users the extent to which they can rely on the
proposed translation or terminological information (de Barra-Cusack 2014). The
recent surfacing of the question of trust, including its quantification as a score, is,
in part, an attempt at protection against technology-driven pragmatism based on
the ‘fit for purpose’ principle, allowing the use of translations varying in quality.
Furthermore, it is no surprise that ‘creating trust’ is also considered to be one of the
key challenges in successful crowdsourcing and its governance (Jain 2010).
This discussion illustrates the way in which translation today is shaped by
technology in micro and macro contexts, while raising human issues such as the
erosion of translator autonomy and the fundamental question of trust. These less
tangible, yet critical, issues are considered in the light of crowdsourcing next.
Research on crowdsourcing and translation crowdsourcing

First named and conceptualized by Jeff Howe (2006), ‘crowdsourcing’ refers to
the action of taking a job to ‘an undefined, generally large group of people in the
form of an open call’. Unlike outsourcing or ‘offline’ forms of user/consumer
participation solicited for product or service developments, crowdsourcing is
novel in terms of the way in which the Internet functions as a worldwide virtual
net to capture the ‘cognitive surplus’ (Shirky 2010) of people online matching
their declared talent to specific skills being sought. Since Howe (2006, 2008), an
increasing number of publications have appeared on crowdsourcing from different
disciplinary interests such as ‘human computation’ (e.g. von Ahn et al. 2008) in
the field of computer science, or ‘distributed innovation’ (e.g. Lakhani 2008) in
business studies, both of which highlight ‘distributed human intelligence’ as the key
idea behind crowdsourcing (Brabham 2013). This is evident in Brabham’s (2013:
xix) definition of crowdsourcing as ‘an online, distributed problem-solving and
production model that leverages the collective intelligence of online communities
to serve specific organizational goals’. This highlights the ‘collective intelligence’
of the participating crowd, which relates to the key idea of the ‘wisdom of the
crowd’, albeit when there is a ‘sufficient diversity’ in the crowd (Surowiecki 2004).
Furthermore, Brabham (2012: 407) maintains that participating crowds are mostly
not amateurs, but ‘largely self-selected experts’, citing crowdsourcing examples
such as the US Patent and Trademark Office (USPTO) and the US Federal Transit
Administration, in which the proposed tasks require specific expertise. While the
degree of expertise sought will depend on the type of crowdsourcing in question, the
call for human intelligence seems to be the key concept, as in Amazon’s Mechanical
Turk (AMT). The AMT is designed to facilitate what they call ‘human intelligent
tasks (HITs)’ that cannot be reliably achieved by computers.
Hossain and Kauranen (2015) present an up-to-date literature review of
crowdsourcing across different disciplines. Drawing on 346 relevant academic journal
articles and conference papers published between 2006 and 2014, they extracted the
top keywords that portray the current picture of crowdsourcing. The key concepts
30 Minako O’Hagan
commonly found in the literature, in order of frequency, were ‘social’, ‘web’,
‘innovation’, ‘open’, ‘information’, ‘human’, ‘online’, ‘community’, ‘mechanical’,
‘collective’ and ‘networks’ (Hossain and Kauranen 2015: 5–6). The authors suggest
that crowdsourcing is overtly aligned with concepts such as social media and
online participation, Web 2.0 technologies and open innovation, while collective
intelligence was also noted as a frequently discussed issue (Hossain and Kauranen
2015: 6). This indicates that crowdsourcing evolves around human intelligence
as opposed to purely computer-based solutions. A description of crowdsourcing
synthesized earlier by Estellés-Arolas and González-Ladrón-de-Guevara (2012: 197)
on the basis of more than 40 different interpretations found in academic publications
does not specify crowdsourced tasks as HITs; rather, it defines crowdsourcing as
‘a type of participative online activity’, involving ‘the voluntary undertaking of a
task’ (Estellés-Arolas and González-Ladrón-de-Guevara (2012: 197). However, the
authors stress the fact that it ‘always entails mutual benefit’ (Estellés-Arolas and
González-Ladrón-de-Guevara (2012: 197), which implicitly suggests that the task
is ultimately rewarding to the participant, who may or may not be paid. Indeed,
the type and level of expertise required for the task sought in crowdsourcing will
depend on the given initiative, but it primarily seeks to have human input. Nakatsu
and colleagues (2014) in turn propose a classification of crowdsourcing based on
task characteristics, according to: (1) task structure (well-definedness); (2) task
interdependence (solo or collaboration work); and (3) task commitment (level of
commitment related to the complexity of the task). This taxonomy again indicates
a broad range of tasks being targeted by crowdsourcing rather than specifying them
as HITs. However, the literature indicates that crowdsourcing privileges human
ntributors to fill the deficit left by a purely computational means.
Hossain and Kauranen (2015) further indicate seven key applications of
crowdsourcing on the basis of the reviewed literature: idea generation; micro-
tasking, open-source software (OSS); public participation; citizen science; citizen
journalism; and wikis. In particular, they attribute translation crowdsourcing to
‘micro-tasking’, which in turn is defined as ‘a system in which users [contributors]
can select and complete small tasks for monetary or non-monetary rewards’
(Hossain and Kauranen 2015: 12), citing the aforementioned AMT as an example.
Furthermore, they surmise that translation crowdsourcing is a suitable means to
obtain ‘high quality translations from non-professionals’ (Hossain and Kauranen
2015: 12). Although otherwise comprehensive, Hossain and Kauranen’s (2015)
literature review refers to only two papers on translation crowdsourcing, one of
which is Zaidan and Callison-Burch (2011), who discussed a computational means
for relatively easily generating high-quality translation from crowdsourced, ‘non-
professional’ translations. This conveys only a partial view on translation, while
contributing to the characterization being made elsewhere of translation as suited
to micro-tasks (e.g. Brabham 2013). Such a notion is at odds with the inherent
nature of translation as a whole3 as well established in translation studies.
Crowdsourcing remains a relatively new research topic in translation studies,
with various different terms used synonymously, as mentioned earlier. Key themes
in scholarly articles written with translation crowdsourcing as the main focus
include: implications for new translation entrants (O’Hagan 2009, 2011; Cronin
2010; Fernandez-Costales 2013); the motivation behind voluntary participation
in translation initiatives (O’Brien and Schäler 2010; McDonough Dolmaya 2012;
Dombek 2014; Olohan 2014); translation quality (Zaidan and Callison-Burch 2011;
Drugan 2013; Jiménez-Crespo 2013; McDonough Dolmaya 2014); translation
ethics (Drugan 2011; McDonough Dolmaya 2011); collaborative workflows
(Anastasiou and Gupta 2011; Ambati et al. 2012); and creativity derived from open
collaborative work modes (Kittur 2010). There is also a cluster of studies from
a translation business perspective comparing crowdsourcing with conventional
LSP models (Kelly et al. 2011), as well as an investigation of translators’ working
conditions in paid crowdsourcing (Garcia 2015). Translation crowdsourcing has
also been approached in the context of major natural disaster management (Munro
2010; Sutherlin 2013) from an organizational perspective focused on logistics and
communication, and has been investigated by computational linguists interested
in the rapid development of translation technology applications that are workable
in such scenarios (Ambati et al. 2012).
The picture that emerges of this dynamic phenomenon from the literature
review suggests that the impetus for crowdsourcing lies in the search for
‘human intelligence’. This can be compared to the direction of technologization
of translation as observed in professional settings, where, in a sense, the trend
is towards the curtailment of translator autonomy. By extension, this can be
considered to be limiting the ‘human intelligence’ of translators, with the risk that
they may not perform to their full capacity. The application of the crowdsourcing
model in translation is considered unproblematic in literature outside the domain
of translation studies proper, in which it is assumed that translation is essentially
a micro-task that does not require specialized skills (Eagle 2009). By comparison,
translation scholars direct their attention towards issues more related to ‘the rise
of the amateurs’, investigating the motivation, competence, quality, processes
and ethics of new entrants into the practice. In this way, the limited scope in
the studies conducted on translation crowdsourcing in other disciplines than
translation is apparent. At the same time, translator agency is something that
seems to be overlooked in discussions of this topic in translation studies. This
places the research question concerning the role of technologies, in turn focused
on translator autonomy and trust, more clearly in context.
The next section briefly introduces the analytical framework of ANT, before
presenting a case study based on FTC.
Understanding the role of technology and relations

between actors in a translation crowdsourcing
network using actor–network theory
Actor–network theory
This study needed a lens through which to view the role of technologies in
interaction with human participants in emerging translation practices. Often used
32 Minako O’Hagan
in understanding how humans and non-human artefacts interact in science and
technology studies, ANT sees the world as a network of relations, and treats agency
as distributed and networked equally between human and non-human actors (Latour
1987, 2004). Actor–network theory is often used to shed light on factors relating to
the success or failure of a given technology in gaining hegemony through relations
formed between actors in the given network based on the key concept of ‘translation’ –
albeit with a different meaning from that commonly understood in translation
studies. In its earlier form, ANT was known as ‘sociology of translation’: Callon
and Latour (1981: 279) refer to translation as all kinds of ‘negotiations’, ‘persuasion’
and ‘violence’ in relation to ‘participant interests’ which may associate or dissociate
actors in a network. In ANT, ‘translation’ denotes the actor’s interpretation of his
or her role and objectives, which forms the basis of his or her actions. According
to Latour (2004), a ‘network’ comprises nodes and links whereby individual actors
are ‘enrolled’ and play different roles according to their differing interests. In such
a heterogeneous network, actors are potentially agents of change through their
own ‘translation’, which is often understood to be transformative and stands in
opposition to ‘diffusion’, which is ‘transfer without distortion’ (Latour 2004). In
ANT, the final achievement of the given goal is subject to actors’ translations and to
relations that develop with other actors, which can be studied only using empirical
methods, because the roles themselves do not reveal such information a priori.
The fact that the inception of translation crowdsourcing is rooted in the online
networked world aligns well with the focus of ANT on connections and relations
that develop between actors in a network. In particular, the underlying structure
of translation crowdsourcing built on Web 2.0 as a ‘social web’ stresses the social
dimension embraced by such networks as their distinguishing characteristic,
seeking an analytical framework with which to address both technological and
social aspects, referring to the role of technology embedded in broader contexts.
Furthermore, as opposed to professional translation, whose actors are likely to
adhere to a more predictable interpretation of their roles often bound by norms,
the ad hoc and organic nature of a translation crowd sits well with ANT’s
acceptance of instability and unpredictability in the formation and stabilization
of actor–networks. In translation crowdsourcing, in which the actors are in the
main non-professional translators, translation norms are less likely to prevail. In
the translation studies context, Buzelin (2005: 205) highlights the merits of ANT
as able to provide ‘a better idea of who participates in the translation process, how
they negotiate their position, and of how much and where translators, in practice,
comply with or contest norms’. The present study seeks, in particular, to capture
emergent relationships that may have developed unexpectedly, which will later be
used to consider translator autonomy and trust.
The next section applies ANT to the FTC initiative as a brief case study.
Unpacking Facebook Translation crowdsourcing according to ANT

Actor–network theory allows the researcher to identify the actors in the identified
network and to describe their relationships during the process of the stabilization of
the network. The theory encourages empirical studies of the detailed interactions
between the actors within the network. In this case, such data were drawn from
doctoral research presenting an original empirical study conducted by Dombek
(2014), under the supervision of this chapter’s author. Dombek’s work makes
detailed ethnographic data available from a population of Polish Facebook ‘user-
translators’, in a study that combines netnographic and remote observational
methods, forming a rich contextual inquiry into the nature of user-translator
motivation in FTC. Its key focus therefore differs from that of the present study,
as do the theoretical frameworks employed. The Polish presence in Facebook and
FTC represents a significant network, and, for the purpose of the present study, the
Polish example is treated as representative of other FTC communities.4 In addition,
the present study draws on the publicly available patent information (Wong et al.
2014) that describes the technical details of the Facebook Translations platform
for which Facebook filed for a US patent in 2007.5
Facebook’s application of translation crowdsourcing was one of the earliest
such attempts made scalable to a potentially large number of participants, with
FTC reportedly having involved 400,000 volunteers by 2010 (van der Meer
2010). Facebook successfully appealed to potential contributors with the goal of
making Facebook available to non-English speakers through the donation of (a
small amount of) their time and linguistic talent. Despite the fact that Facebook
is a profit-making organization making use of free labour, this approach seems to
have been readily accepted by contributors, possibly because Facebook’s services
are offered free to users (Dombek 2014). Furthermore, Facebook’s argument
for opting for user-based translation was the users’ familiarity with the social
networking environment, including the type of language used in the user interface
(UI), making users better able to translate Facebook specific concepts than
professional translators unfamiliar with social networking and Facebook (Losse
2008). The proposal entailed a clearly defined task: to translate short strings, such
as text fragments of the UI elements of the Facebook website in English. With
such strings ranging from one word to a sentence in length, the specific approach
taken by FTC was to present translation work as a micro-task. The contributor
could translate as little as one word without any obligation to do more and within
any time limit. Dombek’s (2014: 197) surveys of Polish user-translators indicated
no more than two hours being spent in a single session. Her in-situ recorded
observations with seven study subjects further established that the majority of
strings worked on incurred less than a minute of translator time per string (Dombek
2014: 218). Facebook insisted that its main interest in soliciting users’ help was to
strengthen its user base and to build user communities, rather than any financial
motivation (Losse 2008). It further argued that a large investment had been made
to develop the dedicated translation crowdsourcing platform called Translations.
The basic top-down nature of the organization of FTC can be seen in the highly
specific nature of the platform, which automates the translation-related workflow.
Figure 2.1 shows the potential actors associated with the FTC networks in
a schematic form, with the Facebook Translations platform (FTC platform) as
the key technological system supporting the FTC initiative. For the purpose of
34 Minako O’Hagan
Figure 2.1 A potential network of actors linked to the FTC platform
this analysis, Facebook management is represented as the Facebook FTC team

that is responsible for the overall operation of FTC, including the platform. In
the recruitment of volunteer translators, Facebook made an open call, initially
targeting its existing members and actively soliciting their participation in the
project. By far the majority of Facebook users do not participate in FTC, but some
members in this group are implicitly linked to FTC contributors by virtue of using
their translations. Ultimately, the Facebook FTC team aims to serve Facebook
users through the FTC effort, thereby linking these two actors. Volunteers who
are not Facebook members and who wish to participate in the initiative need to
register with Facebook to access the FTC platform. This shows a degree of top-
down control, recognized as a characteristic of crowdsourcing structure, which
is often combined with a bottom-up structure that is likely to emerge from the
initiative of participants (Brabham 2013). Facebook’s registered users are then
able to participate in any of the translation-related activities, including editing
Figure 2.2 Facebook Translations platform configuration

Source: Adapted from Wong et al. (2014)
and voting on input by other FTC contributors. Importantly, Facebook also

engaged an LSP for professional translation input, including the validation of
some of the user-based translations. Those contracted translators are, in turn,
likely to be linked to a wider body of professional translators, some of whom
may also be users of Facebook. Both FTC contributors and the LSP are likely to
use additional external translation tools and resources beyond the FTC platform.
Similarly, non-contributing Facebook users may apply tools such as online MT
as their own translation solution, which is also incorporated into the Facebook
environment (Drugan 2013: 171).
The role and interests of each actor enrolled in the FTC network are clearly
different. For example, an FTC contributor has a range of motivations, but mainly
wants to see Facebook made available in a given language (Dombek 2014); the
professional translators contracted by Facebook have a specific goal based on
their briefing by Facebook, while also being indirectly associated with FTC
contributors through assessment of some of their output. In turn, the wider body
of professional translators may be associated with FTC as Facebook users, with
some of them potentially contributing to FTC, whereas others disagree with the
FTC approach and are unlikely to participate in FTC.
Figure 2.1 presents a simplified macro view of FTC actor–networks to which a
more detailed micro-analysis focused on the role of technologies within the FTC
platform can be added, as shown in Figure 2.2.
36 Minako O’Hagan
The Facebook patent document (Wong et al. 2014) explains how the FTC
platform is configured to allow the translation workflow to be optimized for
Facebook’s social networking environments. The Facebook translation module
is made up of five sub-modules, which are sequentially linked, as shown in
Figure 2.2. The identification module uses members’ registration information
to associate them with the relevant language and displays required translations
in that language. The input module displays the source text and receives its
translation. It also makes available translation resources, such as a discussion
board (subsequently replaced by ‘community group pages’), a glossary and a
style guide in the FTC contributor’s UI, while incorporating TM-like features by
displaying similar existing translations. It can also show previous submissions
made by the current contributor for further editing. The discussion board, which
was the contributors’ main means of interaction in the case study presented in this
chapter, was designed to allow collaboration between contributors. Subsequently
added functionality that allowed translators to embed screenshots and videos
(Dombek 2014: 63) further underlines the importance given by the FTC team
to contributor collaboration, and to this resource in particular. The Input module
is linked to the source phrases store and the translated phrases store. The
submitted translation then goes to the voting module, which collects votes on the
given translation. These votes are then assigned specific weights in the weighting
module, which in turn applies a credibility coefficient and a translation controversy
index, as appropriate. The coefficient is derived from the voter’s track record on
the basis of his or her previous submissions, as rated by others. Furthermore,
votes from the member’s ‘friends’ (in its specific Facebook meaning) may be
more heavily weighted than those who are not friends, with the latter vote not
being counted at all in some contexts. The controversy index, in turn, indicates
the level of disagreement about a given translation based on voting patterns; a
similar number of up and down votes, for example, would indicate controversy.
The index also takes into account the weighting on the given vote. Both of these
modules are then stored in the translation ratings store. Favourably received and
weighted translations are then displayed in the presentation module, together with
alternatives, in some cases.
Discussion
The foregoing, mainly descriptive, attempt to apply the ANT framework to FTC
begins to address the roles and relations that emerge among the actors enrolled in
the FTC network. In this section, we delve deeper into the human issues that arise
in the specific technologized context of FTC, focusing on the issues of autonomy
and trust.
Autonomy
Crowdsourcing discourse is often flavoured with democratization and user
empowerment (Howe 2008; Brabham 2012), with the literature highlighting
the ‘mutual benefit’ to the giver and the taker of the given activity (Estellés-
Arolas and González-Ladrón-de-Guevara 2012). In a similar vein, FTC can be
seen as matching Facebook’s translation needs with its then rapidly expanding
international user base, comprising users with relevant skills eager to participate
in an activity that would enhance the social network under development. On the
basis of self-determination theory (SDT) (Deci and Ryan 2008), Dombek (2014)
demonstrated how, to some extent, FTC fulfilled the intrinsic motivation of the
contributors by meeting their need for a sense of autonomy, competence and
relatedness to others – where ‘autonomy’, in the context of SDT, refers to the
perception of an activity as self-determined and internally regulated. This sense
of self-determination seems evident, given the non-binding nature of the task,
in which participants can take a highly selective approach. This was confirmed
by Dombek (2014: 190), who also found a high turnover to be common among
FTC contributors, who often treated the task as a ‘short-term engagement’. At
the same time, Dombek’s detailed examination of the Polish FTC contributors
revealed a negative role played by the technology, which created an obstacle to
fully satisfying the needs of contributor autonomy.
Three main themes emerged from Dombek’s (2014: 160–87) netnography:6
linguistic and terminological issues in the Polish translations; problems with the
translation module; and a lack of communication and feedback from Facebook.
Linguistic and terminological issues involved existing translations that contributors
inherited when they started, and are related in part to the high turnover of FTC
contributors mentioned earlier. Some of the incorrect translations were found to
be related to the use of placeholders, or ‘tokens’, which are used to avoid having
to translate recurring segments in full each time. As is well known in software
localization however, this technique is liable to cause grammatical problems,
especially with highly inflected languages such as Polish. Tokens are handled
in the input module of the translation module (Figure 2.2), to which contributors
had no internal access. The issue led to a plea for help to the Facebook team from
the FTC contributors; the members would often show with concrete evidence that
the cause of mistranslations was the use of tokens, most of which they could not
control. These appeals for assistance were typically not answered by Facebook,
however;7 the discontinuous line between the FTC contributors and the Facebook
FTC team in Figure 2.1 indicates this breakdown. This particular connection in
the FTC network was essentially non-existent, with Facebook relegating its input
to the FTC platform and, in practice, expecting all translation-related issues to be
resolved among the community members, with potentially valuable sources of
user feedback remaining underutilized by Facebook.
The application of ANT to FTC helped to highlight the problematic connection
between the FTC team and FTC contributors as a weak link, which may have cost
Facebook a long-term sustainable translation network when management failed to
recognize this particular vulnerability as significant in their network. To achieve
the goal of FTC in the longer term, some enhancement of the interaction between
FTC contributors and Facebook is needed, as per Dombek’s (2014: 273–4) final
recommendation.
38 Minako O’Hagan
The self-perception survey conducted by Katan (2011) indicated that translation
is at risk of falling short of being considered as a profession, given the diminished
level of autonomy it revealed. Katan (2011) further related these findings to the
argument currently being advanced in the translation studies literature, whereby
activism is inherent in translation, implying translator autonomy and agency. Katan
(2011: 84) goes as far as suggesting that translation theory may be out of touch with
reality. The increasingly technology-mediated environment in which professional
translation is embedded prompts the question of why translation technology is not
designed to better accommodate user autonomy. In the non-professional setting
of FTC, the freedom granted in terms of contribution patterns may initially satisfy
a degree of autonomy, yet ultimately their inability to control the governing
technology (in this case, the translation module) may lessen contributors’ sense of
autonomy. This, in turn, seems to lead to frustration and attrition among otherwise
eager participants, who initially show deep engagement in the translation task
(Dombek 2014). This could suggest that the sense of autonomy created in FTC is
an illusion – one that is at odds with the ‘user empowerment’ manifesto.
Trust
In relation to the question of trust, one of the major issues in translation crowdsourcing
is, arguably, that of quality, given that contributors are self-selected. From a
translation quality assessment perspective, Drugan (2013: 174–5) categorizes
FTC’s approach as a hybrid model combining top-down and bottom-up methods,
in which users vote on the quality of translation, which may also be subject to
evaluation by the professional LSP. As is widely reported in the literature (O’Hagan
2009; Jim nez-Crespo 2013; Dombek 2014), the explicit use of a user voting
mechanism is characteristic of Web 2.0 platforms, but is uncommon in professional
translation settings. An examination of the FTC platform illustrates the way in
which Facebook used a number of approaches to assure quality. The algorithms
applied in the voting and weighting modules in the translation module (Figure 2.2)
quantify the trustworthiness of the submitted translations, as also suggested by the
terminology, with a ‘credibility’ coefficient forming ‘quality scores’ in FTC (Wong
et al. 2014). Similarly, the use of a ‘controversy’ index suggests an automatic
avoidance of translations that cause divided opinions among the voters, which are
hence not considered entirely trustworthy. Furthermore, the way in which a vote
from a ‘friend’ is made to count more heavily than one from a voter unconnected
to the translation contributor (Wong et al. 2014) comes across as somewhat at odds
with the objectivity usually expected of quality assessment, yet perhaps reveals the
specific character of Facebook as ultimately a social networking site that aims to
expand ‘friend’ networks. Such interpretations lead one to consider the possibility
that the notion of trust, as applied in quality assessment in FTC, is very specific to
this environment and is an engineered one. Drugan (2013: 175) alludes to a paradox
of trust in crowdsourcing whereby the organizer assigns more trust to its contributors
by the very act of taking its translation request to an open crowd (or a more
restricted community, as can be argued in the case of FTC), whereas a conventional
professional translation model based on pre-vetted translators provides for more
checks in the form of quality assurance, so that professional translators appear to
be less trusted. At the same time, there is evidence of distrust on the organizer’s
side in crowdsourced translation, given that the ethical guidelines prepared for
crowdsourced contributors incorporate measures to counter subversive behaviour
more explicitly than is the case with their professional counterparts, as shown by
Drugan (2011). In fact, the FTC example seems to indicate that the premise of trust
as a force behind positive relationships is breached, surprisingly on the organizer’s
side, creating the weak link in the actor–network.
Conclusions
This study set out to investigate the role and impact of technology in translation
crowdsourcing in the particular case of a high-profile Facebook initiative.
Motivated by the wish to understand the impact of increasingly pervasive
technology on human translators, the study was framed using ANT, and focused
on translator autonomy and trust from a sociological perspective. Actor–network
theory helped to facilitate the analysis, highlighting emergent roles and relations
among the key actors enrolled in the FTC actor–network and eliciting problem
areas. The study suggested that the FTC platform was mainly designed for
intra-group interaction among the contributors, at the expense of desired links
with the FTC team. This seemed to curtail the sense of autonomy of the group
of dedicated FTC contributors, because their first-hand discovery of significant
problems with the technological platform led nowhere, with the only possible
avenue to solutions being appropriate interventions by the Facebook FTC team.
Such experiences reinforced the perception of FTC as principally a one-way, top-
down structure, not unlike proprietary CAT environments, while the intended
bottom-up structure was poorly facilitated. Hence the contributors’ need for a
sense of autonomy seemed to become frustrated and it is likely that their sense of
trust in the organizers was negatively affected by the lack of reciprocity on their
parts. Such findings seem to be in conflict with the oft-made claim that Web 2.0
environments facilitate user empowerment.
In turn, the FTC organizer’s trust in the quality of the translations submitted by
the FTC contributors was based on quantification by the algorithms embedded in
the translation module. The algorithms displayed an intricate design accounting,
among other things, for the past performance of the FTC contributors and allowing
differentiation between votes on quality cast by others based on the voters’
relationship with the submitter of the translation. In this system, the seemingly
objective quantification of trustworthiness of the submitted translations is clearly
skewed, since it depends on connections between the translator and the voter,
and specifically on whether their relationship is that of ‘friend’ or not. Such
approaches can be taken as an attempt to mimic innate human factors, but in the
specific context of FTC, thereby engineering an algorithmic trust network.
This study found that ANT provides a useful analytical framework for eliciting
social impacts of technologies by making it possible to observe their role in
40 Minako O’Hagan
relation to other actors involved in a real-life network. As has been found in other
applications of ANT (e.g. Cresswell et al. 2010), this lens helps us to recognize the
complexity and the fluidity of social reality associated with the implementation of
technologies that frequently result in non-linear and non-causal paths (Cresswell
et al. 2010). However, as highlighted by Cresswell and colleagues (2010), ANT
is not without criticism: it fails to take into account such background factors as
intentions, morals, learning, culture and previous experiences associated with
human actors, or attributes of technological objects relating to their historical path
that shape their current role. Furthermore, as highlighted by Buzelin (2005) and,
more recently, by Risku and Windhager (2013), translation studies research has
not yet fully exploited ANT as a useful framework of analysis and therefore the
present study has proceeded in the absence of prior examples. The study has only
scratched the surface of the potential significance of translation crowdsourcing
and the application of ANT as an analytical framework has remained introductory.
Despite such weaknesses, the study has shown the potential merit of pursuing a
line of inquiry that sees the relationship between technology and translation as a
fluid series of interactions and connections.
The question of translator agency has become a concern for translation
scholars at a time when translators are seen to have an ever-deepening relationship
with technology. But, compared to overt questions related to non-professional
translators entering into the field, less tangible factors such as translator autonomy
and trust can be overlooked, despite their potentially significant consequences.
Current professional settings suggest a lack of strong interest on the part of
technology designers, developers or implementers in considering translator
autonomy. This is an underlying issue that goes hand in hand with the issue
of trust. The emerging practice of translation crowdsourcing can be used as a
mirror to consider the future implications of the current one-sided approach to
technology; translators are at risk of becoming passive automata that are simply
‘told’ to use a predetermined tool. To mitigate such risks, and given the historical
origin of some CAT tools, developers of translation technologies are best advised
to consider translator autonomy as a critical design benchmark, thereby leading
to a trustful and sustainable relationship between tools and their key users. This
process could prompt translators to take a more active role in shaping technology
and to technology developers gaining an insight into how human translators work
with technology. If translation crowdsourcing acts as a catalyst to deliver this key
message, it will prove to have been a most useful social experiment, revitalizing
translation in the technological era. It is hoped that this study will encourage more
in-depth empirical research, aimed at gaining a situated understanding of how the
relationship between technology and translation is evolving, and how it can best
be shaped to serve the work of translators, be they professional or otherwise.
Notes
1 For further information, see http://www.lionbridge.com/solution-categories/enterprise-
crowdsourcing/ [accessed 12 March 2016].
2 See http://www.proz.com/forum/business_issues/153668-professional_translators_against_
crowdsourcing_and_other_unethical_business_practices.html [accessed 12 March 2016].
3 It must be noted here that text fragments presented as strings without context are
common in software localization, which may be amenable to the view of translation as
based on the sentence as processing unit.
4 One source suggests that 11.8 million people, or 31 per cent of the Polish population,
were active users of Facebook in Poland in 2014 (http://was-sg.wascdn.net/wp-content/
uploads/2014/01/Slide034.png). Some 6,000 Polish volunteer translators reportedly
contributed at least one translation between 2008 and 2012, and, as of November 2012,
there were 71 participants in the Polish translator community page (Dombek 2014:
64–5), representing the most dedicated group of user-translators.
5 As indicated in Dombek (2014), Facebook Translations is a dynamic environment, as is
Facebook itself, with periodic changes being introduced. However, we argue that the key
concepts of the platform are captured in the original patent application document and
subsequent changes are deemed not to affect its fundamental configuration.
6 Dombek’s study was based on data from the initial Polish FTC discussion board from
28 March 2008 to 26 September 2011, contributing more than 900 on-topic individual
posts, and the subsequent group page exchanges between 13 October 2011 and 31 May
2012 (Dombek 2014: 160).
7 It is possible that the level of attention from Facebook differs according to the relative
importance attributed to the given language community from the perspective of
Facebook’s business interests. However, given the scale of the language communities in
relation to the Facebook FTC team, it seems reasonable to assume that FTC is designed
primarily to run by itself, assuming no or few ongoing inputs from Facebook.
References
Abdallah, K. 2012. Translators in Production Networks: Reflections on Agency, Quality
and Ethics (Dissertations in Education, Humanities and Theology). Joensuu: University
of Eastern Finland. Available at: http://epublications.uef.fi/pub/urn_isbn_978-952-61-
0609-0/urn_isbn_978-952-61-0609-0.pdf [accessed 12 March 2016].
Ambati, V., S. Vogel and J. Carbonell. 2012. ‘Collaborative Workflow for Crowdsourcing
Translation’. Paper presented at CSCW’12, Seattle, WA, 11–15 February. Available
at: https://www.cs.cmu.edu/~jgc/CollaborativeWorkflowforCrowdsourcingTranslation
ACMCoCSCW2012.pdf [accessed 12 March 2016].
American Translators Association (ATA). 2009. ‘Head of Largest Professional Translators’
Organization Blasts LinkedIn CEO for “Thoroughly Unprofessional Practices” ’. Press
release, 30 June. Available at: http://www.atanet.org/pressroom/linkedIn_2009.pdf
[accessed 12 March 2016].
Anastasiou, D., and R. Gupta. 2011. ‘Comparison of Crowdsourcing Translation with
Machine Translation’. Journal of Information Science 37(6): 637–59.
Bowker, L. 2005. ‘Productivity vs Quality? A Pilot Study on the Impact of Translation
Memory Systems’. Localisation Focus 4(1): 13–20.
Brabham, D. 2012. ‘The Myth of Amateur Crowds: A Critical Discourse Analysis of
Crowdsourcing Coverage’. Information, Communication & Society 15(3): 394–410.
Brabham, D. 2013. Crowdsourcing. Cambridge, MA: MIT Press.
Buzelin, H. 2005. ‘Unexpected Allies: How Latour’s Network Theory Could Complement
Bourdieusian Analyses in Translation Studies’. The Translator 11(2): 193–218.
Callon, M., and B. Latour. 1981. ‘Unscrewing the Big Leviathan: How Actors
Macrostructure Reality and How Sociologists Help Them to Do So’. In K. D. Knorr-
Cetina and A. V. Cicourel (eds). Advances in Social Theory and Methodology: Toward
42 Minako O’Hagan
an Integration of Micro- and Macro-Sociologies. Boston, MA: Routledge & Kegan
Paul, 277–303.
Christensen, C. M. 2000. The Innovator’s Dilemma. New York: Harper Business.
Cresswell, K. M., A. Worth and A. Sheikh. 2010. ‘Actor–Network Theory and Its Role
in Understanding the Implementation of Information Technology Developments in
Healthcare’. BMC Medical Informatics and Decision Making 10: 67. Available at:
http://www.biomedcentral.com/1472-6947/10/67 [accessed 12 March 2016].
Cronin, M. 2010. ‘The Translation Crowd’. Revista Tradumàtica 8: 1–7.
de Barra-Cusack, F. 2014. A User-Oriented Study of Metadata in focal.ie. PhD thesis.
Dublin City University. Available at: http://doras.dcu.ie/20237/ [accessed 12 March
2016].
Deci, E. L., and R. M. Ryan. 2008. ‘Self-Determination Theory: A Macro Theory of
Human Motivation, Development, and Health’. Canadian Psychology 49(3): 182–5.
Désilets, A. 2007. ‘Translation Wikified: How Will Massive Online Collaboration Impact
the World of Translation?’. Opening keynote speech at Translating and the Computer
29, London, 29–30 November. Available at: http://www.mt-archive.info/Aslib-2007-
Desilets.pdf [accessed 12 March 2016].
Dombek, M. 2014. A Study into the Motivations of Internet Users Contributing to
Translation Crowdsourcing: The Case of Polish Facebook User-Translators. PhD
thesis. Dublin City University.
Dragsted, B. 2004. Segmentation in Translation and Translation Memory Systems: An
Empirical Investigation of Cognitive Segmentation and Effects of Integrating a TM
System into the Translation Process (PhD Series 5). Copenhagen: Samfundslitteratur.
Drugan, J. 2011. ‘Translation Ethics Wikified: How Do Professional Codes of Ethics and
Practice Apply to Non-Professionally Produced Translation?’ Linguistica Antverpiensia
10: 97–111.
Drugan, J. 2013. Quality in Professional Translation: Assessment and Improvements.
London/New York: Bloomsbury Academic.
Eagle, N. 2009. ‘txteagle: Mobile Crowdsourcing’. In N. Aykin (ed.). Internationalization,
Design and Global Development. Berlin/Heidelberg: Springer, 447–56.
Estellés-Arolas, E., and F. González Ladrón-de-Guevara. 2012. ‘Towards an Integrated
Crowdsourcing Definition’. Journal of Information Science 38(2): 189–200.
Fernandez-Costales, A. 2013. ‘Crowdsourcing and Collaborative Translation: Mass
Phenomena or Silent Threat to Translation Studies?’ Hermēneus 15: 85–110.
Garcia, I. 2015. ‘Translation and Technology: Cloud Marketplaces – Procurement of
Translators in the Age of Social Media’. Journal of Specialised Translation 23: 18–38.
Gaspari, F. 2014. ‘Online Translation’. In Sin-wai Chan (ed.). Routledge Encyclopaedia of
Translation Technology. London: Routledge, 578–93.
Hossain, M., and I. Kauranen. 2015. ‘Crowdsourcing: A Systematic Literature Review’.
Strategic Outsourcing: An International Journal 8(1): 2–22.
Howe, J. 2006. ‘The Rise of Crowdsourcing’. Wired 14(6). Available at: http://www.
wired.com/wired/archive/14.06/crowds.html [accessed 12 March 2016].
Howe, J. 2008. Crowdsourcing: Why the Power of the Crowd Is Driving the Future of
Business. London: Random House.
Jain, R. 2010. ‘Investigation of Governance Mechanisms for Crowdsourcing Initiatives’. In
Proceedings of Americas Conference on Information Systems. Atlanta, GA: Association
for Information Systems, 557–63.
Jim nez-Crespo, M. A. 2013. Translation and Web Localization. London/New York:
Routledge.
Karamanis, N., S. Luz and G. Doherty. 2011. ‘Translation Practice in the Workplace:
Contextual Analysis and Implications for Machine Translation’. Machine Translation
25(1): 35–52.
Katan, D. 2011. ‘Occupation or Profession? A Survey of the Translators’ World’.
In R. Sela-Shefy and M. Shlesinger (eds). Identity and Status in the Translational
Profession. Amsterdam/Philadelphia: John Benjamins, 65–87.
Keen, A. 2007. The Cult of the Amateur. New York: Doubleday.
Kelly, N. 2009. ‘Freelance Translators Clash with LinkedIn over Crowdsourced
Translation’. 19 June. Available at: http://www.commonsenseadvisory.com/Default.
aspx?Contenttype=ArticleDetAD&tabID=63&Aid=591&moduleId=391 [accessed 12
March 2016].
Kelly, N., R. Ray and D. DePalma. 2011. ‘From Crawling to Sprinting: Community
Translation Goes Mainstream’. Linguistica Antverpiensia 10: 75–94.
Kenny, D. 2011. ‘Electronic Tools and Resources for Translators’. In K. Malmkjaer and K.
Windle (eds). The Oxford Handbook of Translation Studies. Oxford: Oxford University
Press, 455–72.
Kittur, A. 2010. ‘Crowdsourcing, Collaboration and Creativity’. XRDS 17(2): 22–6.
Lakhani, K. R. 2008. ‘InnoCentive.com’. Harvard Business School Case. Cambridge, MA:
Harvard Business School.
Latour, B. 1987. Science in Action. Cambridge, MA: Harvard University Press.
Latour, B. 2004. ‘How to Talk about Body? The Normative Dimension of Science Studies’.
Body and Society 10(2–3): 205–29.
Losse, K. 2008. ‘Achieving Quality in a Crowd-Sourced Translation Environment’.
Keynote presentation at the 13th Localisation Research Conference Localisation4All,
Dublin, 2–3 October.
McDonough Dolmaya, J. 2011. ‘The Ethics of Crowdsourcing’. Linguistica Antverpiensia
10: 97–111.
McDonough Dolmaya, J. 2012. ‘Analyzing the Crowdsourcing Model and Its Impact on
Public Perceptions of Translation’. The Translator 18(2): 167–91.
McDonough Dolmaya, J. 2014. ‘Revision History: Translation Trends in Wikipedia’.
Munro, R. 2010. ‘Crowdsourced Translation for Emergency Response in Haiti: The Global
Collaboration of Local Knowledge’. Keynote presentation at the AMTA Workshop on
Collaborative Crowdsourcing for Translation, Denver, CO, 31 October.
Nakatsu, R. T., E. B. Glossman and C. L. Iacovou. 2014. ‘A Taxonomy of Crowdsourcing
Based on Task Complexity’. Journal of Information Science 40(6): 823–34.
O’Brien, S. 2006. ‘Eye Tracking and Translation Memory Matches’. Perspectives: Studies
in Translatology 14(3): 185–205.
O’Brien, S., and R. Schäler. 2010. ‘Next-Generation Translation and Localization: Users
Are Taking Charge’. Paper presented at Translating and the Computer 32, London,
18–19 November. Available at: http://doras.dcu.ie/16695/ [accessed 12 March 2016].
O’Hagan, M. 2009. ‘Evolution of User-Generated Translation: Fansubs, Translation
Hacking and Crowdsourcing’. Journal of Internationalization and Localization 1(1):
94–121.
O’Hagan, M. 2011. ‘Introduction: Community Translation – Translation as a Social
Activity and Its Possible Consequences in the Advent of Web 2.0 and Beyond’.
Linguistica Antverpiensia 10: 1–10.
Olohan, M. 2014. ‘Why Do You Translate? Motivation to Volunteer and TED Translation’.
44 Minako O’Hagan
Perrino, S. 2009. ‘User-Generated Translation: The Future of Translation in a Web 2.0
Environment’. Journal of Specialised Translation 12: 55–78.
Pym, A. 2013. ‘Translation Skill-Sets in a Machine Translation Age’. Meta 58(3): 487–503.
Risku, H., and F. Windhager. 2013. ‘Extended Translation: Sociocognitive Research
Agenda’. Target 25(1): 33–45.
Shirky, C. 2010. Cognitive Surplus: Creativity and Generosity in a Connected Age.
London: Allen Lane.
Surowiecki, J. 2004. The Wisdom of Crowds: Why the Many Are Smarter than the Few.
London: Little, Brown.
Susam-Saraeva, S., and L. Pérez-González (eds). 2012. The Translator 18(2). (Special
Issue on Non-Professionals Translating and Interpreting: Participatory and Engaged
Perspectives.)
Sutherlin, G. 2013. ‘A Voice in the Crowd: Broader Implications for Crowdsourcing
Translation during Crisis’. Journal of Information Science 39(3): 397–409.
van der Meer, J. 2010. ‘Where Are Facebook, Google, IBM and Microsoft Taking Us?’ 2
August. Available at: https://www.taus.net/think-tank/articles/translate-articles/where-
are-facebook-google-ibm-and-microsoft-taking-us [accessed 12 March 2016].
von Ahn, L., B. Maurer, C. McMillen, D. Abraham, and M. Blum. 2008. ‘reCAPTCHA:
Human-Based Character Recognition via Web Security Measures’. Science 321(5895):
1465–8.
Wong, Y., S. M. Grimm, N. Vera, M. Laverdet, T. Y. Kwan, C. W. Putnam, J. Olivan-
Lopez, K. P. Losse, R. Cox and C. Little. 2014. Community Translation on a Social
Network. Patent No. US8751213 B2. Washington, DC: USPTO. Available at: https://
www.google.com/patents/US8751213 [accessed 12 March 2016].
Zaidan, O. F., and C. Callison-Burch. 2011. ‘Crowdsourcing Translation: Professional
Quality from Non-Professionals’. In Proceedings of the 49th Annual Meeting of
the Association for Computational Linguistics: Human Language Technologies.
Stroudsburg, PA: Association for Computational Linguistics, 1220–9.
Zhao, Y., and Q. Zhu. 2014. ‘Evaluation on Crowdsourcing Research: Current Status and
Future Direction’. Information Systems Frontiers 16(3): 417–34.
3 ‘I can’t get no satisfaction!’
Should we blame translation technologies
or shifting business practices?
Matthieu LeBlanc
Introduction
As we know, translators have been working with computer-assisted translation
(CAT) tools, in highly technologized work environments, for a number of years
already (e.g. Kenny 2011). The list of CAT tools keeps growing and very few
translators – be they freelancers or salaried employees – can afford to shun
such tools as electronic dictionaries, term banks or translation memory systems
(TMs), for example. The implementation and widespread use of translation tools
have undoubtedly brought about changes to the way in which translators work
(translation processes, workflows, etc.). While many of the tools translators use
have allowed them to become more efficient – by quickly accessing information
that would have otherwise taken hours to locate – other tools have provoked
mixed reactions. Such is the case with TMs. Although TMs have many undeniable
benefits (Bowker 2002; Kenny 2011), they have nonetheless caused some concern
among translators (Garcia 2007; Kenny 2011; LeBlanc 2013). However, what
seems to have unsettled translators the most is not so much the tool’s inherent
design (e.g. the fact that some TMs encourage text segmentation), but more
so the shifts in administrative and business practices that have, in some cases,
accompanied TM implementation. In some instances, TMs have led translation
services and translation service providers (TSPs) to impose certain guidelines that
have caused some disquiet among translation professionals. In the eyes of many
translators, some of the new guidelines – most notably, those pertaining to the
establishment of productivity requirements and the enforced recycling of previous
translations – represent a radical departure from what was done beforehand, and,
more importantly, may have an effect on translators’ professional autonomy and
their overall professional satisfaction.
These are precisely the issues to be addressed in this chapter. More specifically,
we shall examine how these shifts in practices – which were enabled by TMs – have
had an impact on the work of translators and on their professional satisfaction. To
do so, the chapter will draw on data collected through ethnographic observation in
three Canadian TSPs. The focus of the chapter will not be on the overall benefits
and drawbacks of TMs (see LeBlanc 2013 for an analysis of the advantages and
disadvantages of such tools), but rather on some of the business and administrative
practices that have, in those TSPs, followed TM implementation. We will look
46 Matthieu LeBlanc
closely at how translators have experienced those changes, how they have reacted
to them and how they see their profession evolving in the years to come. In short,
the chapter will be focusing on the human aspects of technology integration as
they are tied to shifting practices.
The rest of this chapter is structured as follows. First, it provides some details
on the workplaces in which the study was conducted and on the methodology
that was used to collect the data. It then provides a brief overview of the benefits
and drawbacks of TMs as perceived by translators, after which it describes
the changes that translators have witnessed in the TSPs under study – more
specifically, in relation to the establishment of productivity requirements and
to enforced recycling of translations. The ensuing sections examine closely the
translators’ reactions to those changes and comment on the overall effects of those
new practices. To illustrate the views expressed by translators – and, to a lesser
extent, management – excerpts are provided from the semi-directed interviews
that were conducted with participants. The chapter ends with a discussion of the
findings and some concluding remarks.
Workplaces and methodology
The translation service providers

The study focused on translators and their relationship with technologies. It aimed to
look into the professional realities of translators at work in three different translation
environments (TSPs) located in Canada. The TSPs – the names of which cannot
be made public under Canada’s policy on ethics in research involving humans –
are either private translation firms (sometimes known as ‘translation agencies’) or
public/para-public translation services. They provide translation services either to
a variety of clients (in the case of private firms) or to different public or para-public
entities. They also offer a variety of services, although the bulk of the work done
is translation per se, more specifically in Canada’s official languages, English and
French. The TSPs, to which we will refer as ‘TSP-A’, ‘TSP-B’ and ‘TSP-C’ for the
purposes of this chapter, can be profiled briefly as follows.
TSP-A employs 55–60 employees, 28 of whom are language professionals

(translators and revisers). Translation memory technology was implemented
at TSP-A in 2006. Most of the translation (general/administrative, technical
and specialized) is done from English into French, although there are a few
translators who specialize in the French-to-English combination. In total,
100 hours were spent on site at TSP-A and 17 semi-directed interviews
conducted, as well as seven half-day sessions of shadowing translators at
their individual workstations.
TSP-B is slightly smaller than TSP-A. In total, it employs 22–25 employees,
including 14 language specialists (translators and revisers). TSP-B introduced
TM technology in 2008. As is the case with TSP-A, most of the work performed
at TSP-B is translation from English into French and the texts are very similar
‘I can’t get no satisfaction!’ 47
in nature to those done in the other TSPs (general/administrative, technical
and specialized texts). The same number of hours were spent on site here
(approximately 100 hours), and, in total, 19 semi-directed interviews were
conducted and seven half-days of shadowing translators at their workstations.
The third and last firm, TSP-C, is the largest of the three. It employs 75–80
people, but only 36 are translators and revisers. TSP-C introduced TMs earlier
than the two other TSPs, in 2001. Again, the bulk of the work is general/
administrative, technical and administrative translation. More than 100 hours
were spent on site, which included 16 semi-directed interviews and seven
half-days of shadowing.
Methods used
All in all, about 300 hours were spent on-site with the TSPs, engaging in different
activities. By using ethnographic research methods – interviews, observations,
participation – to study translators in their own environment (Koskinen 2008; Flynn
2010; Hubscher-Davidson 2011), it was possible to gain a better understanding
of the inner workings of each TSP – that is, its mandate, its operational structure,
the nature of the work, the clients, etc. Because of its versatility as an approach,
ethnography has the signal merit of allowing the exploration of translation
practices ‘in the broadest sense’ (Flynn 2010: 116). It can also allow researchers
to perhaps ‘better tap less tangible aspects of the translation process’ (Hubscher-
Davidson 2011: 2), including those less tangible aspects of translators’ use of
translation tools. The observations of translators at work, at their workstations,
and the semi-directed interviews with both translators and management provided
contextual information on the use of tools – both translation technologies and
more traditional tools – as well as on translation practices, working conditions,
professional satisfaction, and so on.
The use of translation technologies in the TSPs under study

All three TSPs under study made extensive use of translation technologies,
alongside more traditional tools. In the course of their duties, translators used a
variety of tools and resources (electronic dictionaries, term banks, etc.), but the
tool that had most drastically changed the way in which they work over the years
leading up to the study was the TM. Although TMs had been introduced earlier
at TSP-C, all three TSPs now made extensive use of TMs and translators had,
over the years, become very much at ease with the tool. In fact, contrary to what
is sometimes believed, the vast majority of translators were receptive to new
technologies and had adapted well to the introduction of the TMs, now the central
tool in the translation process. All three TSPs required their translators to use
TMs, irrespective of text type. (The only exceptions were highly confidential texts
or texts in the field of advertising, with which TMs are of little help.) The TSPs
relied on the use of TMs to increase translator productivity and to provide better
terminological (and, to a lesser extent, stylistic) consistency. At TSP-B and TSP-C,
48 Matthieu LeBlanc
specific guidelines concerning the use of TMs had been issued to translators, as we
shall see shortly.
Benefits and drawbacks of TMs: an overview

LeBlanc (2013: 6–10) describes the main advantages and disadvantages of TM use
in the three TSPs under study here. From a translator’s perspective (based on a set of
statements made by translators in interviews), the main advantages were that TMs:
• help to increase productivity;

• help to improve consistency (terminology, phraseology);
• eliminate uninteresting and repetitive work (e.g. updates, manuals);
• can be used as searchable databases (parallel corpus); and
• can have a pedagogical function (e.g. sharing of solutions, subject knowledge
repository).
The main disadvantages, on the other hand, were that:
• TMs change the translator’s relationship with the text (segmentation) (see
LeBlanc 2014 for a detailed analysis of this phenomenon);
• TMs are a barrier to creativity;
• TMs make translators lazy and increasingly passive;
• TMs have an effect of the translator’s natural reflexes;
• beginner translators rely too heavily on TMs;
• TMs are sometimes polluted (multiple solutions for one segment or term);
• TMs contribute to error propagation;
• TMs influence productivity requirements for translators;
• translators can be forced to use existing translations; and
• TMs render the translator’s work more mechanical and, when misused, may lead
to deskilling and may have an effect on the translator’s professional satisfaction.
In this chapter, we focus exclusively on the last three disadvantages pointed out
by translators, because they were given only cursory treatment in LeBlanc (2013).
Moreover, not only were mentions of these disadvantages recurrent throughout
the dataset (especially from translators at TSP-B and TSP-C), but also, and more
importantly, they pointed towards a link between technology implementation
and the subsequent introduction of new guidelines by the TSPs. In other words,
these disadvantages did not pertain to the TMs per se (i.e. their inherent flaws as
perceived by translators), but more so to the way in which translators were being
forced to used them. They thus warranted further investigation.
Implementation of TMs and the establishment of new guidelines

At TSP-B and TSP-C, management had progressively implemented a series
of guidelines on the use of TMs by translators. These guidelines were adopted
soon after TMs were implemented, once all translators had had a chance to
become familiar with the tool. More precisely, the guidelines were established by
management, sometimes after consultation with the tool manufacturer. They were
not established in collaboration with translators or revisers. The guidelines were
clearly stated in written policies and procedures made available to all translators,
and discussed with translators at regular staff meetings. In this section, we look at
the two major changes captured in these guidelines that have had an effect on the
translators’ working conditions: the establishment of productivity requirements
and the enforced recycling of translations.
Productivity requirements
It goes without saying that TSPs have invested heavily in TMs to increase
productivity and efficiency. This is the whole idea behind TMs and the TSPs
on which this study focused were quite up-front about this. More precisely,
what TM tools have allowed TSPs to do is to factor in exact and fuzzy matches
when establishing productivity requirements for their translators. The following
sections explain how this is achieved at the three TSPs under study.
TSP-B and TSP-C

At TSP-B and TSP-C, productivity guidelines for translators are very similar.
At TSP-C, the guidelines have been in effect since around 2008; at TSP-B, they
gradually came into effect in 2010. The goal in establishing such guidelines was
to harmonize practices and increase efficiency.
First, all new texts are analysed against the TM. This preliminary analysis
provides translators and management with an idea of the effort the translation will
require. This effort is estimated based on a whether an exact, fuzzy or no match
already exists in the TM in question for segments in the new texts to be translated.
For example, in the case of 100 per cent (full or exact) matches, the translation
effort required is set between 20 per cent and 25 per cent of the effort that would
be required for a segment for which there is no match in memory; for fuzzy
matches (99–75 per cent matches), the effort required is set between 70 per cent
and 75 per cent.1 For ‘new’ segments (0–74 per cent matches), the effort required
is set to 100 per cent. These weighted calculations of effort also take into account
internal repetitions – that is, repetitions within the text to be translated. The results
of this preliminary analysis are provided in a report that is produced for every new
source text; the report clearly specifies the time (in hours and minutes) that will be
allotted to translators to translate such texts.
It must be specified, however, that the weighted calculations are not taken into
account when billing clients – that is, no discounts are passed on to clients based
on the number of exact or fuzzy matches, or internal repetitions.2 It must also be
mentioned that productivity requirements – the number of words to be translated
per hour – have increased at both TSPs over the last two years.3 We look more
closely at the perceptions regarding this new practice shortly.
50 Matthieu LeBlanc
TSP-A
At TSP-A, practices were quite different at the time when fieldwork was originally
conducted. Although TM use was widespread and translators felt very at ease, on
a technical level, with TM tools, the major difference was that management did
not use weighted calculations when setting translator productivity requirements.
Translators had a set number of words to translate per hour, irrespective of what
they may or may not have found in the TM. In other words, the TM was used
simply as a tool to support translation and translators were told to use their
judgement when it came to recycling segments. If the segment were a good fit
and thus reusable, they would be strongly encouraged to use it in their translation;
if not, they were free to start from scratch. What had changed, however, were
productivity requirements: since the introduction of the TM, the average number
of words to be translated per hour had gone up at TSP-A.
That being said, the practice of not taking into account previously translated
segments in establishing productivity requirements was considered by most
translators at TSP-A to be transitional. In fact, at TSP-A, management confirmed
that it was in the process of establishing guidelines very similar to those at
TSP-B and TSP-C, and translators had been made aware that changes were on
the way.
Enforced recycling of existing translations
TSP-B
At TSP-B, guidelines pertaining to the recycling of existing translations were very
specific and had been established with the aim of harmonizing practices throughout
the service. Made available to all translators, the guidelines detail how productivity
is calculated and explain how to handle previously translated segments – that is, full
and exact matches (100 per cent), as well as substantial fuzzy matches (75–99
per cent). The guidelines clearly state that translators are to use matches exactly
as they are retrieved by the TM tool (except if they are deemed unusable). The
rationale for this is that segments that are found in the TM have been translated and
revised by professionals employed by TSP-B, and then delivered to clients. If the
pre-existing translation is deemed unusable, the translator must make this known
to a supervisor before any changes are made to the translation. This approval is, in
fact, required. Managers at TSP-B have confirmed that the aim is to avoid multiple
solutions emerging for one and the same source segment – a situation labelled
‘proliferation’ in the TSP’s policy manuals and by employees.4 In practice, however,
there are situations in which translators reject certain matches without requesting
authorization from a manager, and such ‘transgressions’, as several translators refer
to them, have been the cause of discussions between translators and management.
Translators who insist on not recycling as is are first reminded by revisers or
managers of the policies and procedures in place; the few repeat offenders who
insist on modifying segments for purely stylistic reasons – that is, to improve the
text – have been told that such actions go against established guidelines, are counter-
productive and may be brought up during employee performance assessments.
TSP-C
At TSP-C, the guidelines are almost identical to those in place at TSP-B. Simply
put, translators must reuse the translations retrieved from the TM (full and exact
matches, as well as substantial fuzzy matches). Enforced recycling is closely
monitored and translators who take it upon themselves to improve a sentence
or a segment, for stylistic reasons, can, in fact, be reprimanded. The aim is, as is
the case at TSP-B, to avoid ‘proliferation’. In cases in which translators consider
that a pre-existing translation from the TM absolutely needs to be altered for one
reason or another, the modification needs to be approved by a senior translator or
reviser, who must then sign off on it. The TM is then updated and the previous
translation for the segment in question is erased from the memory. Every step is
taken to avoid situations in which multiple translations are retrieved from the TM.
TSP-A
At TSP-A, translators are not instructed to reuse existing translations exactly as
they are, at least for the moment. As noted earlier, management is looking into
establishing new guidelines similar to those in effect at TSP-B and TSP-C.
Translators’ perceptions of shifting practices
Productivity requirements and weighted calculations

The increased pressure exerted on translators is a recurring theme in the vast
majority of interviews conducted in the three TSPs. On the one hand, translators
refer to the general increase in productivity requirements – that is, number of words
per hour – that they have witnessed in recent years; on the other, they allude to the
new way of calculating these requirements via weighted calculations (at TSP-B and
TSP-C). One intermediate translator5 at TSP‑B sums things up as follows:
We’ve been using those guidelines for a couple of years now, and they still
haunt me. I would love to spend more time on my translations, but I cannot.
Everything is weighted. [TSP-B] was too quick in establishing those guidelines.
Every second and every hour is accounted for in the report [preliminary analysis].
(TR12B6)
Another senior translator and reviser, ill at ease with the new guidelines, mentions
that this new practice is having an effect on both the younger translators and the
more seasoned ones:
The new productivity requirements [set out in the guidelines] are stressing
out all my translators. No one was subjected to this much pressure when I
first started here at [TSP-B]. It was widely known to be the best place for a
newcomer to acquire his basic training after university. Now we seem to be
focusing exclusively on productivity.
(TR11B)
52 Matthieu LeBlanc
Another common issue raised by translators pertains to the advisability of using
such weighted calculations in establishing productivity requirements. This is at
the core of the debate between many translators and managers. One intermediate
translator at TSP-B calls into question this new practice and worries about its
effect on the quality of the translations delivered to clients:
The new tools have led to new ways of approaching the text [the source text]
and more recently to new ways of calculating productivity. [ . . . ] As you
know [ . . . ], fuzzy matches remain fuzzy matches. You still have to read the
whole sentence, read all that surrounds it, make sure you’re being faithful to
the new source text. Same thing with exact matches. In some cases such as
extremely repetitive texts, this can work, but in many other cases, this is just
wrong. Let me show you [the participant proceeds to show me an example].
(TR02B)
This view is echoed by a beginner translator at TSP-C:
For me, productivity was an issue. I was lagging behind for the first three
months I was here, but for the last six months, I’ve changed my way of doing
things. I’ve stopped racking my brains when it comes to perfect and fuzzy
matches. I just take what’s there, what the TM proposes, because these words
are hardly factored into my required effort. I was told by my reviser to spend
less time on polishing fuzzy matches and to focus more on increasing my overall
productivity. It’s a major problem. And it has an effect on quality, if you ask me.
(TR03C)
Many more translators allude to the fact that the weighted calculations can, at
times, be very misleading and do not always reflect the total effort required to
translate a text. As one reviser puts it:
The ways now used to measure productivity are very deceiving, and it does
make translators react. There needs to be a better solution.
(TR05B)
For others, it is the fact that there is no way of taking into account the level of
difficulty of a translation and the complexity of working around even substantial
matches (75–99 per cent) that is causing growing concern among translators, as
reported by two senior translators:
Machines [the results of the preliminary analysis] are now dictating to the
nearest minute the time and effort required to translate a text. A machine, as
you know, has no clue of the level of difficulty of the text that I’m translating.
And it can’t be expected to understand how complex this whole operation is.
Exact and fuzzy are not necessarily synonymous with good [quality].
(TR01C)
Moreover, some translators wondered who, in the end, benefited from this new
way of establishing productivity requirements. The translator or the TSP?
Yes, this [weighted calculations] is great for the employer. But it’s not so
good for the translator.
(TR13B)
And another participant adds:
We know fully well that the savings are not passed on to the client, so who,
in the end, benefits from this? Are we just down to costs?
(TR01B)
As we see, the new guidelines – that is, weighted calculations – have led
to considerable disquiet among translators at TSP-B and TSP-C, who were
very forthcoming in expressing their concerns. Although some referred to a
certain disconnect with management, it must be said that some members of the
management teams at TSP-B and TSP-C are aware of those concerns. As one of
the managers comments:
We’ve achieved considerable success in integrating the new [TM] tool

throughout [TSP-B] in the last few years. I am however very aware of the
pressures that are exerted on translators. A lot of it has to do with increased
productivity [requirements]. As a former translator myself, I’m sensitive to that.
(MGR02B)
Other comments, however, seem to confirm a certain disconnect between translators

and management:
I don’t see a problem, really. I can’t see how the new tools and the new
ways of doing things have caused translators to become less satisfied at work.
We’re not really asking them to do more. We’re simply asking them to take
into account what has already been done, to recycle when required to.
(MGR01C)
Enforced recycling of existing translations

As previously mentioned, the second shift in practices, closely connected to
increased productivity and weighted calculations, is enforced recycling of existing
segments – more specifically, full and exact matches (100 per cent), including
internal repetitions, as well as substantial fuzzy matches (75–99 per cent).
Although the two practices are closely linked, for the purposes of this chapter,
a distinction is established between them as far as possible, because most
participants distinguish them. What will be presented here are comments made
by translators working at TSP-B and TSP-C, where recycling is enforced.
54 Matthieu LeBlanc
First and foremost, the majority of translators at TSP-B and TSP-C were quite
vocal about this practice. Most had some misgivings, although a few felt that it
was simply logical to recycle when required. The bulk of the comments made
by those who opposed or questioned this practice pertained to the nature of the
practice itself and to its effect on the quality of the final product, as summed up
by the following participants:
The guideline says that since the translations have been done by other
professionals, they are therefore of good quality and reusable. I personally take
issue with that. If it’s not good, it’s not good. As a reviser, I have the authority
to change a segment if I judge that it is of poor quality, but I was told by my
manager to quit touching up existing translations [in the TM] and move on.
(TR03B)
We are indeed subjected to existing translations. It feels very unprofessional

[ . . . ] I haven’t always observed this guideline and it has led to some
disagreements with revisers, who have seen their authority challenged. The
debate revolves around quality. This is quite a shift.
(TR01C)
In the same vein, others worry about the hazards of such a practice, given that
it is impossible to know the conditions under which these first translations were
produced and entered into the TMs. Sometimes, translations are produced by
translators working for the same TSP, but at other locations in Canada, or by
freelancers working for the TSP:
We are required to recycle [existing translations], but we don’t know the

conditions under which these translations [in the database] were produced. We
all know that translators work with increasingly tight deadlines. Were their
texts reread? Revised? Who knows? We can’t just blindly accept what’s there.
(TR13B)
Moreover, some translators point to specific examples of error propagation caused

by recycling and mention that mistakes often remain uncorrected. According to
one intermediate translator:
In the last press release I translated, I noticed a glaring mistake in one of

the 100% matches. Remember, those are the segments that are to be left
untouched, according to our guidelines. It had been in the system for over six
months. How many times was this sentence recycled? How many times was
this error reproduced? This is embarrassing.
(TR16B)
That being said, for a few translators, enforced recycling was not considered overly
problematic, although one commented on how this was a strange requirement for
a professional:
Recycling doesn’t really bother me. But it’s strange for a professional. It’s
strange that I have to get my reviser’s approval to change a segment, don’t
you think?
(TR12C)
Overall comments on the effects of shifting practices

As apparent in the foregoing, shifts in practices have not gone unnoticed by
translators at TSP-B and TSP-C. In this section, we shall examine how translators
perceive the overall changes that they have witnessed over the past few years. It
must be borne in mind that many intermediate and senior translators have known
both the ‘before’ and the ‘after’ – that is, they have worked in conditions that are,
in at least two of the TSPs, drastically different from what they were seven or eight
years ago.7 Particularly prominent, among long-practising translators, are references
to the extent of the changes observed over recent years, as well as the overall impact
of those changes. More precisely, references to ‘loss of autonomy’, ‘professional/
unprofessional’, ‘mass production’, ‘worker status’, ‘satisfaction’, ‘commodity’,
‘decrease in quality’, ‘technician/piecework’ and ‘disconnect with management’
are frequent, and these are the elements that this section aims to highlight.
Loss of autonomy, deprofessionalization and shift in status

When describing the shifts in practices, translators were almost unanimous in
alluding to the loss of autonomy they had suffered over the last years. While this was
even more salient in interviews with senior and intermediate translators, even junior
translators were aware of the extent to which this loss of autonomy had had an effect
on their more senior colleagues. As one intermediate translator/reviser sums it up:
Translation memories were seen as a panacea, the cure for all ills. I, too, was
optimistic. I do like many of the [TM’s] features, but the way we’re using it
now is certainly not making me feel more professional. On the contrary, I feel
we’re slowly being downgraded to worker status – we’re simply producers
of words, and we represent dollar signs to management. Had I known ten
years ago that this is the direction that [TSP-B] were to take, I would have
reconsidered [taking a job there]. [TSP-B] used to be the cream of the crop,
as you know [ . . . ], but this is changing.
(TR17B)
This use of ‘worker status’ and ‘producers of words’ (as opposed to producers
of texts) is a reference to increases in productivity requirements and the practice
of enforced recycling adopted by the participant’s TSP. Several other translators
used similar expressions in describing the new conditions under which they work:
It’s certainly not what I signed up for. There have been so many changes,
even in the last five years. It has become an assembly-line type of work.
56 Matthieu LeBlanc
I always thought that [TSP-B] was synonymous with quality, but it’s no
longer the case. It’s now all about productivity and ways of increasing
it. [ . . . ] We no longer own our texts, and this has really affected my
professional satisfaction. I worry about the profession and about where it’s
headed. It’s like a form of mass production or industrialization. We are now
told to manage risk and accept mistakes.
(TR02B)
Visibly uncomfortable with the changing practices, the translator quoted above
refers to ‘assembly line’, ‘mass production’ and ‘industrialization’, all of which
did not apply to translation even 10 years ago, based on her experience. As she
explains later in the interview, translators then were autonomous professionals,
who, while required to produce a certain number of words per day, still played an
important role in establishing quality standards and productivity requirements,
and whose opinions in all matters linguistic were taken into account. According
to two other senior translators, the shift in status over the last few years is
patently obvious:
[ . . . ] translation is a mere commodity now. No one seems to care about

quality; it’s all about productivity. I mean, the philosophy has changed. Are
we becoming translation technicians? What will newcomers to the profession
think? How will they deal with the piecework? How will they even be able to
sharpen their skills? I’m worried about the direction professional translation
is taking here in Canada, to be frank.
(TR19B)
Well, sometimes, I feel like I’m a junior clerk [laughter]. Translation is not
prestigious anymore, it’s no longer the liberal profession it once was. [ . . . ]
Productivity requirements have had an effect on quality, and all of this seems
fine with management. [ . . . ] I wonder if translation – well translation as we
practise it here at [TSP-C] – will change and attract those who have a good
enough command of languages but are especially gifted with tools.
(TR01C)
While some may argue that translation has never been considered a liberal
profession – alongside law, medicine and engineering, for example – many will
argue that, at least in the Canadian context, translators were afforded much more
freedom than they have now, on the one hand, and that quality was always at the
forefront of the profession, on the other. A common view among interviewees is
that lack of freedom, or decisional power, and increasing disregard for quality are
cause for concern and are detrimental to the profession as a whole.
Moreover, the new practices have led some participants to wonder if, given
the loss of autonomy and increased automation, translation will now attract, as
another translator (TR16B) puts it, ‘mere bilinguals’ who are not affected by the
increased mechanization of the translation process.
Translators’ professional satisfaction
Overall, the above comments made by participants speak to a certain decline in
professional satisfaction among translators, be they juniors, intermediates or seniors.
Many translators have directly alluded to their lack of recognition as professionals:
The nature of the work is changing fast. I really no longer like what I do, as
we’re no longer valued as professionals. We simply work with segments, take
other people’s work, recycle it as is required by management, and deliver it.
It’s a cold task. I am really trying, making an effort to like the type of work
I’m doing. It’s a far cry from the days when clients would consult me for my
linguistic expertise.
(TR06B)
In the same vein, another translator, post-interview, shared her concerns with
respect to the new practices implemented by confirming that the guidelines are
completely undermining her professional satisfaction (TR04B). Others point out
that translators are no longer part of the process – that is, they are no longer
consulted when changes in administrative and business practices are being
discussed. As an intermediate translator commented:
I cannot understand why we, as professionals, have not been consulted, why
we haven’t really taken an active part in the whole process [developing and
implementing the new guidelines]. We are no longer part of the process. And
yet we could provide valuable feedback, as translation professionals. We could
have shown them [management] that it’s not really the tool itself that’s at fault
but the way they’re forcing us to use it. This alone is enough to affect morale.
(TR08B)
Translators at TSP-A: in a league of their own?

In sharp contrast to the translators at TSP-B and TSP-C, translators at TSP-A
had very little to say in this respect, because TM implementation had not yet
had an effect on translation business practices (productivity, recycling). Although
some translators had misgivings about some of features of their TM tool (most
notably, text segmentation) and most alluded to a general increase in productivity
requirements, none referred to a ‘loss of autonomy’ or to a ‘decrease in professional
status’, although many were apprehensive about the changes to come. Nonetheless,
the contrast in practices was obvious, as the following comment illustrates:
I really enjoy working here at TSP-A. It’s very different from the agency I
worked for in France and from TSP-C, where I worked right before coming
here. In those two firms, we had no freedom. The 100% matches were ‘locked
in’ – we couldn’t even touch them – and the TM was practically sacrosanct.
Here, we have so much more freedom.
(TR08A)
58 Matthieu LeBlanc
Discussion and concluding remarks
Together, the excerpts provided in the foregoing sections indicate that
translators working at all three TSPs have witnessed noticeable change with
respect to translation practices as a whole. Translators at TSP-B and TSP-C
have seen considerable changes on many levels that are consistent with the shifts
described by Gouadec (2007), who points towards a marked industrialization of
professional translation. In the three TSPs under study – most notably, at TSP-B
and TSP-C – the markers of industrialization, as defined by Gouadec (2007),
are patent. Among the most visible are ‘process and product standardisation’
(Gouadec 2007: 300), the ‘use of productivity-enhancing tools’ (Gouadec
2007: 300–1) such as TMs and a ‘never-ending quest for productivity gains’
(Gouadec 2007: 308), as well as a certain ‘division of labour and operator
specialisation’ whereby translation is ‘segmented into a number of different
operations’ (Gouadec 2007: 308). As Gouadec (2007: 311–12) points out, this
industrialization of translation has had several impacts, including an effect on
‘all professional practices and attitudes’, as well as a division of labour that
‘deprive[es] translators of overall control over their work, and . . . an overall
vision of the project they happen to be working on’. He adds that they ‘tend to
become operators working on a virtual assembly line’ (Gouadec 2007: 312).
These changes are consistent with what Mossop (2006) has observed in the
Canadian context – more specifically, the Canadian government’s Translation
Bureau. In an examination of the Translation Bureau’s history since its inception
in 1934, Mossop (2006: 1) contrasts ‘the pre-1995 period, when translation was
done for socio-political purposes, with the past 10 years, when the government
appeared to pursue translation more as an employment-and-profit generating
activity in which Canada could do well’. As underlined by Mossop, before 1995,
the Bureau focused on its sociocultural role, especially after Canada adopted
its Official Languages Act 1969, which made English and French its official
languages. Translation was seen as a way of bridging the divide between the
two linguistic communities and was thus a ‘component of the bilingualization
process which . . . sought to bring about cultural change, that is, a change in how
Canadians conceived public life’ (Mossop 2006: 6). Since 1995, however, the
Bureau has adopted a business-like model and would thus have to cover its costs
– a change that bears many of the markers of industrialization (tools, increase
in productivity, etc.), as described by Gouadec (2007). Translation thus became
an ‘economic activity’ (Mossop 2006: 25), which inevitably has a impact on
linguistic output – that is, quality (Mossop 2006: 26).
These changes – this shift toward a more industrial-type activity, combined
with increased automation – can, as other researchers have pointed out, have an
impact not only on the quality of the final product, but also on the translators
themselves. In an article on the human perspective – or the human factors – in
human–machine translation in the Canadian context, Taravella and Villeneuve
(2013: 71) suggest that ‘what must be avoided is considering language
professionals as mere semi-skilled workers that are only expected to press a
button’. In other words, given the major changes that are being observed in the
world of professional translation, there needs to be more focus on the motivation
and well-being of language professionals, and TSPs thus need to ‘innovate in the
way they manage technology and human resources’ (Taravella and Villeneuve
2013: 71). Put simply, TSPs – or the industry as a whole – needs to focus more
on the human aspects of the interaction between translators and technology.
While the tools in and of themselves may be empowering translators in some
regards, the practices that have followed their implementation must to be taken
into account as well. As Marshman (2012: 10) concludes in her study on the
perceptions of control as indicators of language professionals’ satisfaction with
technologies in the workplace:
[L]anguage professionals who perceive an effect on their control over the

work and working environment as a result of technologies tend to feel more
rather than less in control in many important areas of their work, which is
consistent with the fact that those users of technologies overall view these
technologies as assets.
However, when translators perceive a loss of control over their work because
of technologies, they do not tend to perceive technologies in a positive light.
For example, in her study, Marshman (2012: 10) observed that the ‘imposition
of specific tools and discount schemes, along with the obligation to re-use
solutions from resources, led many respondents to feel they [were] less in control
of their work’. She adds that these reservations are linked not so much to the
tools themselves (i.e. TMs), ‘but rather to human factors and policies in tools’
implementation’ (Marshman 2012: 10, emphasis added). This is, in fact, precisely
what was observed in the present study: translators were neither consulted nor
involved in decision making within the three TSPs.
Overall, what we seem to be witnessing in two of the three TSPs under study
are shifting practices that can be explained by the industrial and economic turn
that professional translation has taken in Canada in the last 20 years. In some
TSPs, the changes have taken longer to materialize, but the effects of these ‘turns’
on translation professionals seem palpable, at least in the TSPs under study. While
the study’s findings cannot be extrapolated to other TSPs or other settings, it
does provide some insight on what has changed – or is changing – in some work
environments and on what this means for professional translators, who see a
considerable narrowing of their role. Translators who once regarded their work
as essential in building bridges between linguistic communities are now feeling
increasingly side-lined not so much by technological advances (as suggested by
Garcia 2009: 210), but by the new practices that TSPs – both public and private –
have established through the use of technology. The fact that translators at TSP-A,
who had yet to witness those changes, were largely uncritical of the practices in
place at their TSP is in itself quite revealing.
This also causes us to reflect on the status of professional translators in
the Canadian context. Although the present study is limited to three medium-
sized TSPs and is certainly not meant to be representative of the Canadian
60 Matthieu LeBlanc
translation community as a whole, it does point towards a certain shift in status
as well. Once regarded by many participants as a profession offering a certain
autonomy and recognition, translation seems to be, according to the majority of
participants at two of the three TSPs, moving towards a more industrial future,
with less autonomy and recognition. This goes beyond the scope of the research,
which did not set out to measure the status of translators per se. However,
the question of status does come through, directly or indirectly, in the data,
which raises the question of whether we are, indeed, at a crossroads. Further
research on the status (similar to the study carried out by Dam and Korning
Zethsen 2010, for instance) and working conditions of translators in Canada
would allow us to have a better understanding of how translators perceive
their status. For example, do translators consider themselves lower autonomy
professionals (LAPs) or higher autonomy professionals (HAPs) (Katan 2009)
– or somewhere in between? Is translation perceived as a profession or as an
occupation? Are we witnessing power shifts (Garcia 2007)? What role have the
implementation and widespread use of technologies played in this regard? Have
they affected practices in other translation environments? Are we witnessing
a certain ‘deprofessionalization’ (Garcia 2009)? Will more and more ‘craft
translators’ have to ‘go industrial’, as suggested by Gouadec (2007)? What are
the marked differences between salaried translators and freelance translators
(which were not included in the present study)? And what role do training
institutions have to play (in which regard, see Kenny 2007, and Marshman and
Bowker 2012, on the importance of encouraging critical thinking skills about
tools in the curriculum)?
Finally, what we also need, perhaps more urgently, are further investigations
of the relationship between human issues and translation technologies. For
example, to what extent are translators pointing the finger at technologies for
their dissatisfaction when, indeed, it is the business practices implemented by
TSPs that are the underlying cause of their dissatisfaction? On a more ideological
level, how do managers of TSPs and designers of tools view the translation
process as a whole? How does this differ from the way in which translators
and language professionals view translation and their work? In what ways is
this relationship conflictual? Similarly, when technologies are integrated in the
workplace, how can translators be better integrated in the process and in the
development of policies and procedures? From what we see, translators seem
to play a limited role in this respect. And finally, when technological change
results in increased dissatisfaction among language professionals, what voice
– collective or individual – do translators have to express their misgivings and
doubts? Do professional associations have any collective power? These are
only some of the questions that have been raised by this study of translators
at work and it is clear that further investigations will need to be undertaken
to better understand the potential effect of translation technologies – and, of
course, the ensuing practices adopted by TSPs – on both translation practices
and translation professionals.
Notes
1 The precise weighting for exact and fuzzy matches varies from one TSP to the other.
2 This is not necessarily the current practice in all TSPs, as many pass on discounts to
clients.
3 The TSPs under study – even though they remain anonymous – prefer that any information
pertaining to productivity (number of words translated per day) not be mentioned in the
researcher’s presentations and publications. Having said that, the average number of
hours worked by translators varied between seven and eight per day.
4 Indeed, individual client TMs are cleaned periodically by support staff to avoid
proliferation.
5 For the purposes of this study, junior translators are considered to have between nil and
five years’ experience (34 per cent of translators interviewed); intermediate translators,
between five and ten years’ experience (20 per cent of translators interviewed); and senior
translators, upward of ten years’ experience (46 per cent of translators interviewed).
6 In this chapter, the code TR indicates ‘translator’; MGR is used for ‘manager’. The last
letter – A, B or C – indicates the TSP by which the participant is employed; the intervening
number differentiates between translators or managers employed by the same TSP.
7 Almost two-thirds of translators/revisers interviewed had known the ‘before’ and ‘after’.
References
Bowker, L. 2002. Computer-Assisted Translation Technology: A Practical Introduction.
Ottawa, ON: University of Ottawa Press.
Dam, H. V., and K. Korning Zethsen. 2010. ‘Translator Status: Helpers and Opponents in
the Ongoing Battle of an Emerging Profession’. Target 22(2): 194–211.
Flynn, P. 2010. ‘Ethnographic Approaches’. In Y. Gambier and L. van Doorslaer (eds).
Handbook of Translation Studies, Vol. 1. Amsterdam: John Benjamins, 116–19.
Garcia, I. 2007. ‘Power Shifts in Web-Based Translation Memory’. Machine Translation
21(1): 55–68.
Garcia, I. 2009. ‘Beyond Translation Memory: Computers and the Professional Translator’.
Journal of Specialised Translation 12: 199–214.
Gouadec, D. 2007. Translation as a Profession. Amsterdam/Philadelphia: John Benjamins.
Hubscher-Davidson, S. 2011. ‘A Discussion of Ethnographic Research Methods and
Their Relevance for the Translation Process’. Across Languages and Cultures: A
Multidisciplinary Journal for Translation & Interpreting Studies 12(1): 1–18.
Katan, D. 2009. ‘Occupation or Profession: A Survey of the Translators’ World’.
Translation and Interpreting Studies 4(2): 187–209.
Kenny, D. 2007. ‘Translation Memories and Parallel Corpora: Challenges for the
Translation Trainer’. In D. Kenny and K. Ryou (eds). Across Boundaries: International
Perspectives on Translation Studies. Newcastle: Cambridge Scholars, 192–207.
Kenny, D. 2011. ‘Electronic Tools and Resources for Translators’. In K. Malmkjaer and K.
Windle (eds). The Oxford Handbook of Translation Studies. Oxford: Oxford University
Press, 455–72.
Koskinen, K. 2008. Translating Institutions: An Ethnographic Study of EU Translation.
Manchester: St Jerome.
Study in Three Translation Agencies and Services’. Meta 59(3): 537–56.
LeBlanc, M. 2014. ‘Les mémoires de traduction et le rapport au texte: ce qu’en disent les
traducteurs professionnels’. TTR: Traduction, terminologie, rédaction 27(2): 123–48.
62 Matthieu LeBlanc
Marshman, E. 2012. ‘In the Driver’s Seat: Perceptions of Control as Indicators of Language
Professionals’ Satisfaction with Technologies in the Workplace’. Paper presented at
Translating & the Computer 34, London, 29–30 November.
Marshman, E., and L. Bowker. 2012. ‘Translation Technologies as Seen through the Eyes
of Educators and Students: Harmonizing Views with the Help of a Centralized Teaching
and Learning Resource’. In S. Hubscher-Davidson and M. Borodo (eds). Global Trends
in Translator and Interpreter Training: Mediation and Culture. London: Continuum,
69–95.
Mossop, B. 2006. ‘From Culture to Business: Federal Government Translation in Canada’.
The Translator 12(1): 1–27.
Taravella, A., and A. O. Villeneuve. 2013. ‘Acknowledging the Needs of Computer-
Assisted Translation Tools Users: The Human Perspective in Human–Machine
Translation’. Journal of Specialised Translation 19: 62–74.
4 How do translators use web
resources?
Evidence from the performance
of English–Chinese translators
Vincent X. Wang and Lily Lim
Introduction
Translators today can take advantage of a wide range of web-based and stand-alone
electronic resources – which are now readily accessible from their workstations – to
facilitate and enhance their professional practice. Their work increasingly benefits
from the use of translation memory (TM) (cf. Laviosa 2011; Olohan 2011; O’Brien
2013), large online monolingual and bilingual dictionaries, specialized glossaries,
comparable and parallel corpora, machine translation (MT) tools, and even social
networking and cloud technology. In terms of English–Chinese translation,
major online English–Chinese dictionary portals incorporate authentic translation
examples, which are retrieved from large Chinese–English parallel corpora that
have been constructed using translated texts in various genres. The dictionary
portals, such as Youdao dictionary (有道詞典), Baidu dictionary (百度詞典), Jiu
Hai (句海), JuKuu (句酷), Ai Ciba (愛詞霸) and Zhongguo Yidian (中國譯典),
usually provide open access, and tend to pop up as top search results. Anecdotal
evidence suggests that these web resources are frequently used by practising
translators; scholarly research has also pointed towards the tangible benefits of
such web resources for translators (cf. Alcina 2008). However, empirical studies
of translators’ actual use of, and interaction with, these resources are still much
needed (cf. Olohan 2011: 343). If this is true of translation in general, then it is
even truer of translation in non-European languages. Most of the existing research
that looks at how translators interact with tools and resources focuses on European
languages, as used in Europe or Canada, for example Désilets and colleagues
(2009) and the prior research cited in that source, as well as more recent work also
conducted in Canada by LeBlanc (2013, this volume) and doctoral work that runs
the gamut from all of the official languages of the European Union (Valli 2013)
to a single majority–minority language pair in Ireland (de Barra-Cusack 2014).
But despite the size of the Chinese translation market, thus far there is very little
published work on how Chinese translators use web resources. A signal merit of
the European-focused sources listed is, however, that they focus on professional
translators in their normal workplaces, thus increasing the ecological validity of
the research. There is no doubt that such workplace-based research would also
be beneficial in the Chinese context, but, given the very early stage of relevant
research in the Chinese-speaking world, there is still much to be gained from more
64 Vincent X. Wang and Lily Lim
‘experimental’ research, in which tasks are designed by researchers to enable
them to observe behaviours in particular scenarios and in which participants can
be selected in very targeted ways. This is important for the purposes of this study,
because we wish to investigate, among other things, the use of web resources by
translators into Chinese from different age cohorts.
This research, then, attempts to describe how English–Chinese translators
utilize web resources to perform a translation task. We investigate two cohorts of
practising translators – a group of younger translators aged 22–25 and a group of
more senior translators aged 45–65 – with the aim of revealing:
1 what web resources the two groups of translators tend to use and with what
frequency;
2 how frequently the translators use web resources to study each difficult word
or word chunk; and
3 how the two groups of translators differ in terms of (1) and (2).
We examine our findings based on Pickering’s (1995, 2008) view of scientific and
technological practice as involving a ‘mangle of practice’, and relate the results of
this study to Pym’s (2011) observation on the use technology in translation that
leads to paradigmatic interruptions of the syntagmatic.
Recent insights into human–computer interaction

In this section, we review Pickering’s (1995, 2008) concept of ‘mangle of
practice’ and Pym’s (2011) proposal that technology disrupts textual linearity –
some new insights into human–computer interaction on which we will draw in
our research – rather than providing a comprehensive survey of the landscape of
human–computer interaction (cf. O’Brien 2012).
‘Mangle of practice’
Andrew Pickering (1995, 2008) proposed the concept of the ‘mangle of practice’
to capture the essence of scientific and technological practice. For him, such a
practice emerges temporally through the interaction between human agency
and non-human (e.g. machine) agency. Human agency and non-human agency
are reciprocally interrelated in the course of this interaction, and the outcome is
unpredictable. He sees this as comparable to the unpredictable shape of laundry
that has been through a mangle, which depends on the ‘interplay’ – that is, the
‘dance of agency’, in Pickering’s terms – between the agency of the laundry and
the mangle. Pickering’s model attaches importance to non-human agency, and
this was intended to remedy the problem of previous models that had tended to
emphasize human agency and downplay non-human agency.
Pickering’s concepts of ‘mangle of practice’ and ‘dance of agency’ were first
applied to the interaction between translators and TM by Olohan (2011). Olohan
perceived that there was much value in Pickering’s account of scientific and
How do translators use web resources? 65
technological practice that attached importance to both human and non-human
agency, and believed that this model would facilitate well-rounded analysis of
human–computer interaction. Olohan’s study follows translators’ postings in an
online technical support forum, revealing the way in which translators interact
with computer-assisted translation (CAT) tool SDL Trados Studio 2009. For
Olohan, the translators’ interactions exemplify the ‘dance of agency’, in which
the translators accommodate the resistance from the TM software and reshape
their own agency in the course of mastering this new technology. She critically
reviews the implementation of Pickering’s conceptualization in her case study
and remains convinced of the value of drawing on his model.
The imposition of the paradigmatic

Pym (2011) examines how technology has transformed translators’ professional
practice. He draws on Saussure’s linguistic terms – ‘syntagmatic’ versus
‘paradigmatic’ – to describe two opposing axes along which translators might
process a text. The syntagmatic axis represents a linear reading of the text – from
beginning to middle to end – while the paradigmatic axis represents the mode
in which translators work on different alternatives for a given segment of the
text. Pym proposed that translators’ use of technology promotes paradigmatic
processing of a text, which causes an interruption or disruption of the syntagmatic
reading of the text. Such interruption and disruption may eventually change the
way in which translators read and work with the text in their translation practice.
With the wide range of technological tools and resources, especially TM, now
available and frequently used in their professional practice, translators are likely
to be engaged in extensive paradigmatic text processing. If this tendency prevails,
as Pym observed, the manner and the quality of translation may be transformed.
The concepts of the ‘mangle of practice’ and the paradigmatic disruption of the
syntagmatic have opened new perspectives for studies of how translators interact
with technologies of various kinds. In the next section, we describe the method
used in one such study designed to investigate translators’ use of web resources.
Method
Informants
The experiment employed two groups of translators as participants – referred
to here as the ‘junior’ and ‘senior’ groups. The junior group of translators
consisted of seven MA students on the translation studies programme (English–
Chinese) in a tertiary institution in Macau, China. The students, four male
and three female, aged 22–25, all obtained their first degrees on the Chinese
mainland, majoring in English or translation studies. Most were in their
second year of study when they participated in the experiment. They had been
extensively trained as translators and had considerable translation experience.
Three of them – Ellen, Nathan and Sue – possessed the prestigious professional
qualifications for translators, 全國翻譯專業資格 (the China Accreditation Test
for Translators and Interpreters, or CATTI), at level two and level three.
The senior group comprised six translators in the 45–65 age range, five male
and one female. Five worked as academic staff in tertiary institutions in Macau,
China, and one in a university in Brisbane, Australia. All taught on translation
programmes and had substantial experience in practising translation in the course
of their professional careers. They were therefore experienced translators and
translator trainers. Four of them also practised interpreting professionally, while
two had Australian translation qualifications – that is, the National Accreditation
Authority for Translators and Interpreters (NAATI), at levels three and four.
All 13 translators were native speakers of Mandarin Chinese. All are referred
to here by pseudonyms.
Procedure
The translators each performed a timed translation task, followed by an interview
with one of the investigators. The translation task involved English-to-Chinese
translation of a 154-word BBC news report about a recent initiative against
Internet piracy in the UK (cf. Appendix). As part of the same task, the informants
were also required to save the web-search history corresponding to the translation
task to the same Word document in which they had produced their translation,
and to fill in a table that summarized their major searches for words and word
chunks. They were able to indicate how the searches assisted their translation task
and to write additional remarks. The follow-up interview was semi-structured,
with the aim of allowing the informants to give a first-hand account of the process
involved in solving translation problems with the aid of web resources. The
investigator posed questions, interacted with the informants, and enabled them to
discuss their specific search techniques and to reveal their perceptions of using
web searches to help with translation. Typical questions in the semi-structured
interviews included the following.
• What are the words or expressions that took you a relatively long time to look
up? What was the outcome of your search?
• Are there any words or expressions that you looked at and retrieved useful
information about relatively quickly? What are they?
• Are there any words or expressions that you made an effort to translate by
yourself ? What are they?
• Did you perform some searches on Chinese word usage?
• Did you use any special techniques for searching? Did you combine English
and Chinese words in your searches?
• Did you use machine translation tools such as Google Translate? If so, how
did they assist your translation?
One of the investigators monitored the translation task performed by the junior
group of translators. It was held in a meeting room of the tertiary institution at
which they studied. All seven informants brought their own laptop computers and
performed the translation task in the Word file circulated by the investigator, while
a printed copy of the file was also provided to each informant (cf. Appendix).
The investigator timed participants’ translations and observed their performance
over their shoulders. The informants submitted their worksheets electronically.
Immediately following the translation task, the investigator interviewed the seven
informants in two groups – one of four and one of three translators. Each group
interview lasted 30–40 minutes; the investigator asked probing questions about
the translation process and each informant took turns to answer. The informants
were able to respond to their colleagues’ remarks and interact with the investigator
to provide elaboration and clarification.
The same investigator met all of the senior translators individually in their
offices or home offices to conduct the same translation task and carried out
individual follow-up interviews with them.
Data analysis methods

We triangulated the information gathered from each informant’s translated text,
search histories and summary table, and contributions to the interviews. We
attached great importance to the translators’ web-search histories, focusing on
(a) the type(s) of word(s) or expression(s) that they looked up most frequently
and (b) whether they tended to process the text along the paradigmatic or
syntagmatic axis, and which web resources they accessed, using which search
techniques.
Results
Characteristics of the junior group’s web searching

We observed a number of distinct characteristics in the web-search behaviour
of the junior group of translators – that is, heavy reliance on web resources,
sophisticated search techniques and paradigmatic interruptions of linearity.
Heavy reliance on web resources

The junior group performed a significantly higher number of web searches than the
senior group (see Table 4.1). The younger translators performed 23.1 web searches
on average, which is more than three times as many as the average number of
web searches (7.5) conducted by the senior translators. All seven junior translators
conducted 10 or more searches to translate the original 154-word text – of the
group, Sue (55) and Andrew (37) were by far the most prolific web searchers.
The junior group retrieved information from a wider range of web resources
than did the senior group (see Table 4.2). At the level of individual translators,
more than half of the younger translators used three or more dictionary portals
and MT tools for this task, while the senior translators tended to use only one
Table 4.1 Comparison of the number of web searches by the two groups of translators
Junior group Senior group
Web searches (n) Web searches (n)
Sue 55 Ruth 17
Andrew 37 Jack 9
Nancy 18 David 8
Nathan 15 Oscar 7
Ellen 14 Andy 3
Carol 13 Ivan 1
Ray 10
Average 23.1 Average 7.5
Table 4.2 Comparison of the use of web resources by the two groups of translators
Web resources Web resources
Sue Baidu dict. (百度); iciba (愛詞霸); Haici dict. Ruth Baidu dict. (百度)
(海詞詞典); Google Translate; Google
Andrew Youdao dict. (有道詞典) Jack ichacha (查查詞典);
Google Translate
Nancy Haici dict. (海詞詞典); Google David Youdao dict. (有道
詞典); Google
Nathan Youdao dict. (有道詞典); Baidu dict. (百度); Oscar Youdao dict. (有道
Wikipedia 詞典); Google
Ellen Cambridge English–Chinese (traditional) Andy Baidu dict. (百度);
dict. Haici dict. (海詞
詞典)
Carol Youdao dict. (有道詞典); Google Translate; Ivan Baidu dict. (百度)
Google
Ray Youdao dict. (有道詞典); Oxford dicts;
Linguee English–Chinese dict.; ichacha
(查查詞典); Google
or two. However, the two groups did exhibit some similarities in their use of
web resources. Both groups favoured Youdao (有道詞典) and Baidu (百度)
dictionaries to look up words and terms, and clearly preferred Google as their
web browser. There were also a range of resources and tools that were common
to the searches by the two groups – for example ichacha (查查詞典), Haici
dictionary (海詞詞典) and Google Translate. The junior group, despite making
use of more varied web resources, did not seem to utilize qualitatively different
web resources from those accessed by the senior group.
The junior group of translators tended to look up a greater number of words
and to launch a greater number of web searches to tackle the challenging segments
of the source-language text. Three words and expressions in the source-language
Table 4.3 Number of searches for key expressions by the junior and senior groups
capped a far cry music tracks capped a far cry music tracks
Sue 10 1 3 Ruth 2 1 1
Andrew 7 4 2 Jack 6 1 0
Nancy 1 1 0 David 0 3 1
Nathan 1 1 1 Oscar 1 1 2
Ellen 3 1 2 Andy 1 1 0
Carol 4 1 1 Ivan 0 0 0
Ray 5 3 0
Average 4.4 1.7 1.3 Average 1.7 1.2 0.7
Table 4.4 Websites visited by the junior and senior groups to search for key expressions
capped a far cry music tracks
Junior group
Sue GT; iciba; Baidu dict.; Haici GT GT; Baidu

dict.; Google; Free dict. × 4 dict. × 2
Andrew Youdao dict. Youdao dict. Youdao dict.
Nancy Haici dict. Haici dict.
Nathan Youdao dict. Youdao dict. Youdao dict.
Ellen Cambridge CE dict. Cambridge CE dict. Google
Carol GT; Google × 2; Youdao dict. Youdao dict. Youdao dict.
Ray Oxford dict.; Youdao dict.; Google CE; Oxford dict.; nil
Google (UK); Linguee × 2 ichacha
Senior group
Ruth Baidu dict. Baidu dict. Baidu dict.

Jack cdict.net; ichacha × 5 GT nil
David nil Oxford dict.; Youdao dict. × 2 Youdao dict.
Oscar Baidu dict. nil nil
Andy Baidu dict. Baidu dict. nil
Ivan nil nil nil
Note: GT = Google Translate; × 4, etc. = used four times, etc.; Cambridge CE dict. = Cambridge
Chinese–English dictionary; Google CE = Google search of Chinese and English words combined
text – capped, a far cry and music tracks – were searched for most frequently
by both groups of translators. The three expressions prompted markedly more
searches in the junior group than in the senior group (see Table 4.3). The younger
translators also employed more web-based tools than the senior translators to
study the problematic words (see Table 4.4). For example, Sue and Ray made use
of six and four tools, respectively, to investigate the word capped.
According to the information gathered from their task performance histories
and follow-up interviews, the junior translators relied heavily on web resources,
which helped them to better understand the source-language text, retrieve
possible translation options and determine whether a particular translation
was commonly used in the target language. They looked up English words
and expressions in bilingual – and sometimes monolingual – dictionaries and
studied examples of the words or word chunks in question, paying particular
attention to the examples that involved contexts similar to those in the text they
were translating.
Their comprehension of the source-language text appeared to benefit most from
their web searches. They stated in the interviews that although the definitions and
examples of the word or word chunk often did not provide an exact equivalent that
they could use in their own translation, the examples they studied on the websites
gave them a better understanding of the meaning(s) of the word or word chunk
and enabled them to work out a suitable translation. For example, they noted that
they had found a rich repertoire of monolingual and translation examples on the
web relating to the word capped, which helped them to understand the various
meanings of the word, although no specific example was perfectly applicable to
their own translation. Their view of the use of these examples was echoed by the
senior group. In addition, the junior translators felt that web searches allowed
them to clarify ambiguities in their comprehension of the source-language text.
Andrew, for instance, specified in the group interview that he had been uncertain
about whether figures in the phrase Latest industry figures suggested referred
to eminent people or numbers, and that he had conducted a search in Youdao
dictionary to resolve this question.
Web searches also contributed to the production of the translation texts. Both
the junior and senior translators noted in the interviews that web-based tools
tended to be powerful and efficient for retrieving the translations of proper names
and terms with commonly used translations. They were able to readily determine
that Digital Economy Act was commonly translated as 數字經濟法案 shùzì jīngjì
fǎ’àn (‘digital economy law or act’) and a far cry was rendered as 相去甚遠
xiāngqù shényuǎn (‘very distant or different from’). Other examples of ready-to-
use translations that they retrieved from the web related to the Labour government,
a compromise and [this three-year] scheme. Sue and Andrew revealed in the
interviews that they had from time to time used web searches in their translation
work to study the usage of Chinese words – for example to check whether a
particular Chinese expression was idiomatic or commonly used, in what context
and by whom. Sue performed several searches for Chinese expressions during the
course of the experiment.
Apart from looking up words or word chunks, the junior translators searched
the web to obtain background knowledge relevant to the translation. For example,
Nathan checked both the Chinese edition of Wikipedia and Baidu regarding
the term of office of the Labour Party in the United Kingdom; this piece of
information helped him to understand and translate with confidence the sentence
When the controversial Digital Economy Act was introduced in the final days of
the Labour government in 2010 [our emphasis], according to his remarks in the
interview.
Sophisticated search techniques
The junior group utilized a range of elaborate techniques that they had developed
to search for the translations they needed, as evidenced by their web search
histories and their interviews with the investigators. One of the techniques they
used most frequently was to Google search a combination of English and Chinese
words. First, they Google searched words or word chunks in English along with
their partial Chinese translations in an attempt to retrieve the full translation of the
words or word chunks from the web. For example, Sue launched a search for the
combined phrase internet piracy 互聯網 in Baidu, in which she added the Chinese
characters 互聯網 hùliánwǎng (‘Internet’) to the word chunk Internet piracy; this
provided search results in which the full translation of the phrase 互聯網盜版
hùliánwǎng dàobǎn (‘Internet stealing (of) copyright’) was easy to find. Similarly,
Ellen did a Google search for the phrase copyright infringement 版權, in which
the Chinese word 版權 bǎnquán (‘copyright’) and the use of double quotation
marks around copyright infringement effectively narrowed down the search results
in which the full translation of the phrase 侵犯版權 qīnfàn bǎnquán (‘piracy’ or
‘copyright infringement’) emerged. A second important function of their search for
a combination of English and Chinese words was to verify their own translations –
specifically, to examine whether their translation co-occurred frequently with the
English word(s) so that it could be considered to be an established translation. For
instance, Carol Google searched the phrase capped for 保留 to verify whether
保留 bǎoliú (‘to keep’ or ‘to maintain’) was potentially a translation equivalent
for capped for; she eventually used 保留 in her translated text, although this is,
in fact, a mistranslation in the given text (cf. ‘Discussion’).
Two translators in the junior group – Sue and Nancy – used Google Translate
to translate long sentences and paragraphs in the source-language text. They
explained in the interviews that the output of Google Translate helped them
to understand the message conveyed by the text, as well as to retrieve useable
translations of technical terms. Sue revealed that her translation of music track
as 曲目 qǔmù (‘list of music pieces’) was a product of Google Translate. She
further expounded on the way in which she ‘played’ with Google Translate by
changing the word order in a given word chunk to obtain the desired translation.
By contrast, only one translator in the senior group – Jack – used Google
Translate to translate paragraphs in the source-language text; he indicated that
this enabled him to understand the message of the source-language text more
readily.
The younger translators also employed techniques to confine their searches to
the desired scope. Ray Google searched capped for a year, site:uk, in which he
used site:uk to retrieve the word chunk capped for a year only from UK websites,
to focus on the instances of its occurrence in British English.
Paradigmatic interruptions of linearity

We observed that the younger translators tended to search on the Internet much
more frequently while performing the translation task; they also use more varied
web resources than the senior translators. The web resources they used and the
search techniques they employed indicate that they devoted considerable time and
effort to studying the various possibilities for translating a given word or word
chunk. Such efforts represent text processing along the paradigmatic axis, which
causes interruption to the linear processing – that is, along the ‘syntagmatic’ axis in
Saussure’s terminology – of the text (Pym 2011: 2). It seems that the way in which
the younger translators read and translated the text lends support to the situation
described by Pym (2011: 2ff), in which ‘technology imposes the paradigmatic’.
Unlike the senior translators, the younger translators spent much more time
looking at alternative translations, which were abundantly available on the web,
than reading the text in-depth in context (on which more will be said under the
heading ‘Discussion’).
Characteristics of the senior group’s web searching

In comparison to the junior group, the senior group demonstrated a tendency to
rely more on their memory, experience and translation competence, and less on
web resources. Relatively speaking, the senior translators read and translated
the text more along the syntagmatic axis than the paradigmatic axis. Within the
senior group, we observed two contrasting tendencies in terms of using web
materials.
More reliance on memory and experience

The senior translators tended to read the source-language text carefully as a
whole and to reflect on the difficult segments, making reference to contextual
information, before translating. This is in stark contrast to the younger translators,
who tended to launch immediately into translation and web searches. This partly
explains why the senior translators performed web searches much less frequently
than the younger translators in our sample. All of the senior translators other than
Ruth (17) searched the Internet fewer than 10 times, and Ivan (1) and Andy (3)
clearly kept web searches to a minimum (cf. Table 4.1).
We observed that the senior translators tended to rely on their memory
and experience in the translation task. They employed their own intuition
and judgement, rather than frequently searching on the Internet to determine
whether their translations were established or plausible. They produced rather
mature and formal prose that was easy for the general reader to understand. One
of the senior translators – Oscar – remarked in the interview that translators
should view their translation from the perspective of others – that is, the general
public – rather than from their own perspective: something he emphasized in his
training of translators.
There were no instances of sophisticated search techniques employed by the
senior group in this sample, and none of the senior translators combined English
and Chinese words in Google searches in our experiment. However, most reported
that they often do so in their daily translation practice.
A divide in the senior group
We observed a divide in the senior group in terms of the use of web materials.
Ivan and Andy performed web searches so infrequently that they seemed to be
at odds with the other four translators, who all readily conducted web searches
when they deemed it necessary (cf. Table 4.1). Ivan and Andy were the only
two translators in this sample who opted to handwrite their translations on
paper rather than typing them on the computer. They can be considered to
largely represent the traditional method of translation using pen, paper and
dictionary, although they did know how to access major web resources such
as Baidu (百度) and Haici (海詞詞典) dictionaries. Both produced skilful
handwriting and rather mature prose in Chinese. However, it appears that they
did not take full advantage of the available web resources to investigate the
precise meaning of certain words. Ivan misunderstood the meaning of capped,
while Andy translated music tracks as the more general 光盤 guangpan (‘CDs’),
even though the meaning of capped and the Chinese translation of music tracks –
for example 單曲 dānqǔ (‘single piece [of music or song]’) and 曲目 qǔmù (‘list
of music or songs’) – would have been available from web resources such as
Baidu dictionary and Google Translate.
By contrast, the other four senior translators spared no effort in utilizing
web resources to assist their comprehension and translation production. For
example, David and Oscar – NAATI-qualified translators at levels three and four,
respectively – made eight and seven web searches, respectively. Oscar even fine-
tuned his translation by Google searching two Chinese phrases – 網上消費 wǎng
shàng xiāofèi (‘consume on the web’) and 在綫消費 zàixiàn xiāofèi (‘consume
online’) – to determine which sounded more idiomatic to Chinese readers.
Discussion
We found that the two groups of translators accessed largely similar types of
web resource; these included: (a) major English–Chinese dictionary portals that
provide authentic translation examples; (b) major web browsers such as Google
and Baidu; and (c) the MT tool Google Translate. The two groups differed mainly
in how frequently they consulted web resources and the range of resources they
employed to look up each difficult word or word chunk. In other words, the two
groups of translators researched words differently in terms of the breadth and
depth of their searches. The junior translators were less confident, basing their
decisions more on search results, while the senior translators relied more on their
intuition and judgement, querying a much narrower range of words. It is not
our intention to judge who performed the task more effectively – the junior or
senior group – because the aim of the study is describe the characteristics of the
web searches they conducted, rather than to evaluate them. However, we believe
that our findings on the different characteristics of these groups deserve more
examination (a) from Pickering’s view of scientific and technological practice,
and (b) with regard to what Pym noted as paradigmatic interruptions of linearity.
The dance of agency in the junior and senior translators
Pickering’s (1995, 2008) concepts of ‘mangle of practice’ and ‘dance of agency’
cast further light on the results of this study. From this perspective, the use of
technology entails interaction – in essence, reciprocal interrelation – between
human agency and technological agency. Web resources, which have become
much more numerous, more varied and more sophisticated for Chinese–English
translators in recent years, are readily accessible to all translators, both junior and
senior. However, the interaction between the junior translators and the technology
tends to differ from that between the senior translators and the technology. The most
plausible reason for this is that the human agency of the two groups of translators
is not identical. The younger translators have less translation experience, and
probably a more restricted repertoire of translation equivalents between English
and Chinese; on the other hand, they have greater familiarity with computers and
the Internet. To cope with our timed translation task, they tended to make greater
use of technology, using sophisticated search techniques to retrieve a larger amount
of information, all in line with their own human agency. Their pattern of interaction
with technology emerges temporally. As they gain in translation experience and
develop a richer mental repertoire of translation equivalents, it is possible that
they will reduce their reliance on web resources and depend more on their mental
translation ability. They thus may move more towards the pattern of interaction
with technology demonstrated by the senior translators. In light of Pickering’s
view of scientific and technological practice, junior and senior translators interact
with the technology in distinct ways according to their individual human agency.
Web resources are designed to provide translators with the information needed
for their translation tasks. However, at times, the resources fail to offer precisely
the information required by translators for a particular translation job – that is,
‘resistance’ arises from non-human agents that human agents need to ‘accommodate’,
in Pickering’s terms. In our experiment, both junior and senior translators found
that the technology was unable to provide an appropriate translation for the word
capped. For example, Sue and Ray had to use several web resources to study
this word (cf. Table 4.4), while Jack read through five consecutive web pages of
ichacha (查查), which contained 50 translation examples of the use of cap. It is
clear that the translators made significant efforts to accommodate the resistance
from the technology. The technology appeared to be ineffective largely owing to
the fact that the word cap is a polysemous word, and can be used as both a noun
and a verb. The abundant translation examples for cap in the web dictionary portals
were not ordered according to each distinct meaning of the word or by grammatical
function (noun or verb). The translators therefore needed to read these ‘unsorted’
examples and deduce possible translation equivalents in different contexts based on
the examples – which requires a significant investment of time and effort and can be
overwhelming. At this juncture, this suggests a means of improving the technology:
if the examples were to be tagged and arranged according to each distinct meaning
of cap, the translators would be able to determine the different meanings of the
word and their translation equivalents more easily – that is, resistance from the
technology would be reduced.
Paradigmatic interruptions of the syntagmatic
In our experiment, the translators’ syntagmatic (or linear) reading of the source-
language text tends to be interrupted by their paradigmatic use of technology –
and this occurs far more frequently in the junior group than in the senior group.
However, there is evidence that even the senior translators in our sample may
not have achieved full linearity in their reading of the source-language text. This
observation relates to the translation of the phrase these messages will be capped
for a year – the most difficult segment in the passage, according to both groups.
Their translations confirmed it as the most problematic segment: most of the
translators mistranslated it (see Table 4.5).
We argue that had the translators carefully read the whole news report on
the syntagmatic axis, they should have been able to rule out the more unlikely
interpretations – such as ‘period of validity’, ‘be blocked’, ‘be kept confidential’ –
and determine the intended meaning. The news report contains a concise lead-in,
Table 4.5 Translations of capped by the two groups of translators
Translation of these messages Gloss of the translation

will be capped for a year
Junior group
Sue 這些資訊發送之後的有效期 The information’s period of validity

為一年 is one year after its delivery.
Andrew 這些資訊的提供將會覆蓋一 The supply of these messages will
年的時間 cover one year’s time.
Nancy 這些警告將只會保持一年 The warnings will hold only one year.
Nathan 這些提醒最多持續一年 These warnings remain up to one
year.
Ellen 這些警告只有一年限期 These warnings have only one year of
time limit.
Carol 這些資訊將會保留一年 This information will be retained one
year.
Ray 上述警示郵件將會保留一年 The said warning emails will be
retained one year.
Senior group
Ruth 這些信息將被封鎖一年 These messages will be blocked one

year.
Jack 上述警告電子郵件在一年之 The said warning emails will be kept
內將被保密 confidential within one year.
David 這些提醒資訊每年不會超過 These alerting messages will not
若干次 exceed a certain number each year.
Oscar 這些短信的上限為一年 These messages’ upper limit is one year.
Andy 上述電郵警告資訊在一年內 The said email-warning information
會作加標記處理 will be treated by adding mark-ups
within one year.
Ivan 這些信息將被覆蓋一年 These messages will be covered one
year.
which states that ‘[f]rom 2015, up to four warnings a year will be sent to households
suspected’ (cf. Appendix). This piece of information is critically important in
helping readers to infer the meaning of the phrase these messages will be capped for
a year, which refers to the fact that the total number of warning messages will be
limited to four in 2015. It appears that all of the translators apart from David simply
overlooked the link between these two segments of text and this was confirmed in
the interviews. Oscar, for instance, understood the meaning of the word capped well,
but did not establish the link with the lead-in. We can observe that the translators
tended to resolve the translation problem regarding capped by conducting extensive
research on the web (cf. Tables 4.3 and 4.4) rather than by examining the source-
language text more closely to discover syntagmatic connections. This observation
lends support to Pym’s (2011) argument that the use of technology tends to cause
interruption to the syntagmatic reading of a text – a theme that is certainly worth
exploring in future studies.
Conclusion
This chapter has investigated how translators use web resources in their translation
practice. Thirteen English–Chinese translators in two age groups – the junior and
senior groups – participated in our experiment. The results were obtained based
on triangulation of data, which included the translators’ translations, web search
histories, written summaries of their searches and responses in the interviews. We
found that the junior translators retrieved information on the web significantly
more frequently than the senior translators. The former used sophisticated search
techniques to access a wider range of web resources, employing relatively
more varied sources to translate the most difficult terms. By contrast, the senior
translators tended to rely more on their memory and experience, and to read
the source-language text in a more syntagmatic (linear) fashion than the junior
translators. Drawing on Pickering’s concepts of ‘mangle of practice’ and ‘dance
of agency’, we found that the two groups of translators tended to interact with
the technology (non-human agency) in distinct ways, in line with their individual
human agency. The results of this study also lend support to Pym’s (2011: 2)
argument that ‘technology imposes the paradigmatic’ – an interesting theme that
deserves particular attention in subsequent future studies.
References
Alcina, A. 2008. ‘Translation Technologies: Scope, Tools and Resources’. Target 20(1):
79–102.
de Barra-Cusack, F. 2014. A User-Oriented Study of Metadata in focal.ie. PhD thesis.
Dublin City University. Available at: http://doras.dcu.ie/20237/ [accessed 12 March
2016].
Désilets, A., C. Melançon, G. Patenaude and L. Brunette. 2009. ‘How Translators Use
Tools and Resources to Resolve Translation Problems: An Ethnographic Study’. Paper
presented at the MT Summit XII – Workshop, ‘Beyond Translation Memories: New
Tools for Translators’, Ottawa, ON, 29 August. Available at: http://www.mt-archive.
info/MTS-2009-Desilets-2.pdf [accessed 12 March 2016].
Laviosa, S. 2011. ‘Corpus Linguistics and Translation Studies’. In V. Viana, S. Zyngier and
G. Barnbrook (eds). Perspectives on Corpus Linguistics. Amsterdam: John Benjamins,
131–53.
Study in Three Translation Agencies and Services’. Meta 59(3): 537–56.
O’Brien, S. 2012. ‘Translation as Human–Computer Interaction’. Translation Spaces 1(1):
101–22.
O’Brien, S. 2013. ‘The Borrowers: Researching the Cognitive Aspects of Translation’.
Target 25(1): 5–17.
Pickering, A. 1995. The Mangle of Practice: Time, Agency, and Science. Chicago, IL:
University of Chicago Press.
Pickering, A. 2008. ‘Preface’. In A. Pickering, K. Guzik, S. Keith, B. Herrnstein,
E. R. Weintraub and A. Franklin (eds). The Mangle in Practice: Science, Society, and
Becoming. Durham, NC: Duke University Press, 1–14.
Pym, A. 2011. ‘What Technology Does to Translating’. The International Journal for
Translation & Interpreting Research 3(1): 1–9.
Valli, P. 2013. Concordancing Software in Practice: An Investigation of Searches and
Translation Problems across EU Official Languages. PhD thesis. Università di Trieste.
Available at: https://www.openstarts.units.it/dspace/handle/10077/8591 [accessed 12
March 2016].
Appendix: The translation task
Here is an outline of the tasks and estimated time allocation:
Tasks Estimated time
A. Translation 45–50 mins

B. Save web-search history 2 mins
C. Summary table 5–8 mins
Total 50–60 mins
A. Translate the following passage including the title (154 words) from English
into Chinese (简体字 or 繁體字). You can write your translation by hand on
paper if you prefer.
Please highlight or underline words that you have looked up in a printed

dictionary or on the web.
Internet piracy warnings
Warning emails are to be sent to people in the UK who are downloading

music and films illegally. From 2015, up to four warnings a year will be
sent to households suspected of copyright infringement.
Anyone who is found to have illegally downloaded material will be
sent an alert offering advice on where to find legitimate sources of
entertainment online.
But these messages will be capped for a year and there will be no
penalties for offenders. It’s a far cry from what the entertainment industry
originally called for.
When the controversial Digital Economy Act was introduced in the
final days of the Labour government in 2010, it included measures to cut
off people’s internet connections for repeated misuse.
This new three-year scheme is a compromise.
Latest industry figures suggest nearly a quarter of all content
consumed online is illegally downloaded, including more than a billion
music tracks within a year.
译文/譯文 (Your translation):
B. The recent web-search history to be copied and pasted (Ctrl + V) to the space
below.
To copy search history from three commonly used web browsers:
Mozilla Firefox: Open History tab > open Show ALL History, select and
copy recent Search history (Ctrl + C)
Google Chrome: Open History tab, select and copy recent Search history
(Ctrl + C)
Internet Explorer: Open View (檢視) tab > open (瀏覽器列) > open (歷程
記錄) > print screen (Ctrl + PrtSc), because Search history cannot be copied
by Ctrl + C.
C. Summary table of major searches (you can add rows)
Words or phrases The Web info was useful Remarks

searched for comprehension e.g. a difficult one
(compr), translation e.g. translations on
(trans), or verifying my the web not suitable,
translation (verify)? I worked out the
(Please delete as translation myself
appropriate)
compr/trans/verify
compr/trans/verify
compr/trans/verify
compr/trans/verify
compr/trans/verify
compr/trans/verify
compr/trans/verify
5 Translators’ needs and preferences
in the design of specialized
termino-lexicographic tools
Alejandro García-Aragón and
Clara Inés López-Rodríguez
Introduction
Translation cannot be understood without technology, nor technology without
translation (Byrne 2012: 3–4). According to Biau and Pym (2006: 18), ‘virtually all
translating is aided by computers’, and it is widely acknowledged that technological
change – for example the widespread use of translation blogs, wikis, open-code
translation software, crowdsourcing, machine translation (MT), translation
memory (TM), cloud-based translation tools, corpora, etc. – has influenced the
way in which both professional and trainee translators work. In some cases,
translation technologies are implicated in workflows in which translators become
just another link in a long chain of intermediaries and they translate fragments of
texts, rather than whole texts. In Biau and Pym’s (2006: 6) words: ‘Translation,
like general text production, becomes more like work with databases, glossaries,
and a set of electronic tools, rather than on complete definitive source texts.’ A
fact that has not changed however, is that translators spend more than half of
their translation time consulting reference works (Varantola 1998; Durán Muñoz
2010, 2011). That said, translators use resources that are not specifically designed
for them (Durán Muñoz 2011: 138).1 This may lead to high ‘lexicographic
information costs’, with translators investing a great deal of effort in searching
for and comprehending lexicographic information and obtaining little or nothing
in return (Nielsen 2008).
It is the duty of lexicographers to identify the relevant function and target group
of a particular dictionary, and then select, adapt and present the data accordingly.
As Bergenholtz and Tarp (2003: 172) put it, ‘lexicographers study – or ought to
study – human activities in order to detect possible needs that can be satisfied by
means of a dictionary’. The termino-lexicographer’s task can be greatly facilitated
by technology; space becomes less of a constraint in an electronic dictionary than
a paper one, for example. Nonetheless, terminographers and lexicographers must
be aware of the dangers of infoxication, because ‘[t]oo much information, like too
little information, can lead to confusion, stress and unnecessary effort on the part
of readers’ (Byrne 2006: 18).
The aim of this chapter is to investigate the information needs of translators
when using lexicographic tools and terminological knowledge bases. To this end,
we explore shifts in translation studies, terminology and lexicography, which
Translators’ needs and preferences 81
recently have paid more attention to the communicative and cognitive needs of
their target users. We also describe the design and results of an online survey on
translators’ termino-lexicographic needs and preferences when translating. This
survey was evaluated by experts in the field, as well as by its respondents, to
validate its contents, items and relevance. Finally, on the basis of these results, we
present the design of a termino-lexicographic prototype for translators working
with atmospheric sciences, MeteoTrad (http://www.meteotrad.com). MeteoTrad
is a practical implementation of termino-lexicography (García-Aragón 2016), a
new general theory of terminology and lexicography that integrates both frame-
based terminology (FBT) (Faber 2012) and the function theory of lexicography
(FTL) (Bergenholtz and Tarp 2003, 2004, 2010).
Translation studies and lexicography: a shift

towards communication and cognition
In translation studies, many researchers and professional translators have tried
to emancipate translation from its apparent dependency on linguistics in favour
of a more autonomous, communicative and interdisciplinary field (Franco
Aixelá 2001: 173). In fact, many researchers point out that translating is not a
linguistic act or process, but an act or process of communication (see, among
others, Toury 1980; Seleskovitch and Lederer 1984: 256; Nida and Taber 1986:
29; Hatim and Mason 1995: 13; Lvóvskaya 1997; Nord 1997; Moya 2004: 42;
Balliu 2013: 92). This is in line with the functionalist approaches to translation,
which take into account all of the participants in the translation process, for
example clients, authors, translators and proofreaders (Strandvik 2013: 330),
with the translation brief as the guiding compass.
Lexicography has undergone the same process of emancipation from linguistics,
especially in the subfield of specialized lexicography, thanks to the modern
theory of lexicographic functions (FTL) (Bergenholtz and Tarp 2003, 2004; Tarp
2005, 2008; Fuertes-Olivera and Tarp 2014). This theory sees terminography
and specialized lexicography as synonyms, and lexicography as an independent
scientific discipline with a specific object of study: dictionaries (Bergenholtz
et al. 2009; Tarp 2012: 323). The main function of a dictionary is ‘to provide
assistance to a specific user group with specific characteristics in order to cover
the complex of needs that arise in a specific type of user situation’ (Bergenholtz
and Tarp 2003: 176). Therefore, dictionaries are seen as ‘utility products’ that
help to solve specific types of problem (Nielsen 2010). This is in consonance
with the functionalist approach to translation, in which translation is seen as the
production of a text ‘in a target setting for a target purpose and target addressees
in target circumstances’ (Vermeer 1987: 29).
According to the FTL, any lexicographic resource should be designed according
to the function(s) it is due to fulfil. A lexicographic function can be defined as
the satisfaction of the specific types of lexicographically relevant need that may
arise in a specific type of potential user in a specific type of extra-lexicographic
situation (Tarp 2008: 81). The FTL identifies communicative functions and
cognitive functions. Communication-related functions can be subdivided into
82 Alejandro García-Aragón and Clara Inés López-Rodríguez
production (writing properly in a given language), reception (understanding
a given language) and translation (producing from one language to another)
(Tarp 2005: 8–9). Cognitive functions are motivated by the need to acquire new
knowledge or to check existing knowledge about a specific topic or a specific
language service provider (LSP) (López-Rodríguez et al. 2012: 59). Cognition-
related functions can be further subdivided into linguistic, encyclopaedic or
specialized data (López-Rodríguez et al. 2012: 59).
Translators, as a target group in lexicography, require both communication-
oriented and cognition-oriented features: ‘In order to successfully achieve
communicative goals, translators need to be provided with knowledge about the
conceptual structure underlying the subject field they are working with’ (León
Araúz et al. 2008: 999). This is one of the premises of a cognitive approach to
terminology known as ‘frame-based terminology’ (Faber 2012), for which ‘an
ideal lexicographic resource for translators should describe (i) the meaning of
words; (ii) their use in context; (iii) their possible correspondences in other
languages; and (iv) their position in the configuration of the mental lexicon’
(Tercedor Sánchez et al. 2013: 182).
Lexicography and translation meet in bilingual and multilingual lexicography,
but there are few resources especially designed for translators (Durán Muñoz
2010, 2011).2 This situation is compounded by the fact that translators spend a
great deal of time consulting dictionaries (Roberts 1997; Varantola 1998; Sánchez
Ramos 2004), while markets are increasingly demanding greater productivity
(Candel Mora 2010: 272). We argue that a dictionary or termino-lexicographic
resource – umbrella terms here for any lexicographic or terminographic tool – that
is aimed specifically at translators should give them what they are looking for in
a fast and effortless way to help them to continue with their translation task. This
would doubtlessly benefit them in terms of time and productivity. What is more,
according to the functionalist theories of translation, both the source text and the
target text are considered an ‘offer of information’, or Informationsangebot (Reiss
and Vermeer 1984: 119), in which not all information has the same value. This
should also apply to the data found in any dictionary: ideally, there would be a
differentiated information menu from which the translator-user would choose the
most suitable or relevant information depending on the brief.
In any case, the quality of the information included in dictionaries and
its organization cannot be neglected, and terminology theories such as FBT
provide reliable bottom-up and top-down methodologies established on solid
cognitive premises that can help to organize and represent the specialized domain
that the dictionary intends to cover. Frame-based terminology is a cognitive
approach to terminology that argues that subject-specific concepts should be
organized in a way similar to how concepts are structured in the human mental
lexicon (León Araúz et al. 2010, 2013; Faber et al. 2011; Faber 2012; Faber
and López-Rodríguez 2012). This can be achieved by using frames through
practical implementations such as EcoLexicon (LexiCon Research Group n.d.).
EcoLexicon is a multimodal knowledge base on the environment that seeks to
meet the cognitive and communicative needs of a wide range of users, from
environmental experts to technical writers, translators and the general public. It
contains around 3,600 concepts and 20,200 terms in Spanish, English, German
and Modern Greek; three new languages – Russian, French, and Dutch – have
recently been added.3 The present authors’ experience with FBT and the FTL, as
well as with other related theories, disciplines and reference works, is the basis
for the conception of termino-lexicography (García-Aragón 2016). This new
interdisciplinary, cognitive, functional approach is the basis for the design of a
prototype that aims to be a termino-lexicographic resource for translators working
in one of the domains of covered by EcoLexicon: Atmospheric Sciences, which
prototype we describe later in the chapter.
Moreover, having in mind that translations, equivalents and texts change during
the transfer process depending on the brief, the ideal lexicographic tool should
provide guidance, as well as contextualized information validated by experts in
the field. Thus definitions should ideally be context-constrained and coherent,
co-texts and usage patterns should be extracted from carefully selected texts, and
images and other multimodal information should be adapted and presented in a
user-friendly interface. Such concerns show that terminology and lexicography
have experienced the same shift towards user-friendliness and user evaluation as
other fields in recent years (López-Rodríguez et al. 2012). These cognitive and
communicative shifts, together with the advent of functionalism in lexicography,
have increased the fuzziness in the boundaries between lexicographic and
terminographic tools for translators as well as for other user groups.
To conclude this section on the convergence of translation studies and
termino-lexicography, we will establish a parallelism between the different
functions in a dictionary for translators (Tarp 2005: 8–9; Tarp 2013, 2014), the
information needs of translators during ‘the translation phase proper’ (Fuertes-
Olivera and Tarp 2014: 68), some translation sub-competences as identified by
Hurtado Albir (2007: 385), ‘the four uncertainties of the translator’ (Duvå et al.
1992: 132) and the items that – according to FBT (Tercedor Sánchez et al. 2013),
as well as to Durán Muñoz (2010: 63–4) – an ideal lexicographic resource for
translators should include. These features are depicted in Table 5.1 alongside
their corresponding ‘routes’ in MeteoTrad.
These four aspects can be used to structure the information included in
a dictionary for translators, which, in turn, would ideally assist users in the
translation process. However, all of these assumptions about the termino-
lexicographic implications and preferences of translators need to be verified by
translators themselves – a step that we cover in the next section.
A survey of translators’ lexicographic needs and preferences
Aims, items and sections of the survey

An online questionnaire was created as an invitation to respondents (professional
translators, lecturers of specialized translation and translation students) to reflect
upon lexicographic resources (worded as ‘dictionaries, encyclopaedias and similar
Table 5.1 Parallelisms between termino-lexicographic functions, translation sub-competences and dictionary items
Functions in Information needs of translators Translation sub- Four uncertainties of Items of an ideal Routes in MeteoTrad
a dictionary during the translation phase competences (Hurtado the translator (Duvå lexicographic resource (García-Aragón 2014b)
for translators proper (Fuertes-Olivera and Albir 2007: 385) et al. 1992: 132) for translators (Durán
(Tarp 2005: Tarp 2014: 68) Muñoz 2010: 63–4;
8–9; Tarp Tercedor Sánchez
2013, 2014) et al. 2013)
Reception Definitions of source language Extralinguistic The semantic content The meaning of words Definitions (cognitive
terms competence of words tasks at the concept
(cultural and level)
domain knowledge)
Production Collocations and fixed Linguistic competence Use of words Use of words in Usage (communicative
expressions; information context tasks at the textual
about orthography, gender, level)
grammar, genre conventions
Translation Equivalents of terms Transfer competence Place of words in the Possible Equivalents
universe correspondences (communicative
of a word in other tasks at the lexical
languages level)
Cognition Subject field background Extralinguistic Subject matter in Position of a word in Knowledge (cognitive
information competence question configuration of tasks at the subject-
(cultural and mental lexicon field level)
domain knowledge)
resources’) in any format or medium (paper, online or other electronic medium).
The aim of the survey was to explore respondents’ termino-lexicographic needs
and preferences when translating. The results could then be cross-checked
against our previous assumptions on their cognitive and communicative needs,
preferences and tasks, and lacunae. The survey was written entirely in Spanish
owing to the fact that the project was first conceived at the University of Granada,
Spain. It comprised 48 questions, divided into an introductory part and three
sections about lexicographic resources. The survey was open to any respondent
who (a) had an advanced knowledge of Spanish and (b) was familiar with the
translation process.4
The questionnaire design followed accepted principles. As López-Rodríguez
and colleagues (2012: 62) stress:
Questionnaire design is crucial when it comes to extracting reliable data and

feedback from any type of user group. If questionnaires, tests, scales or other
measuring instruments do not follow adequate psychometric criteria, any
conclusion extracted from them may be wrong, biased or misleading.
The development of items is the most important phase in the creation of measuring
instruments (Downing 2006), since items are the bricks that form and shape
them. All of the items we included were lexicographically or terminographically
motivated, but also oriented towards the cognitive and communicative needs,
preferences, tasks or lacunae of potential users: professional translators, lecturers
in specialized translation and translation students.
We believe that psychology-based standards are a sound starting point
for our aim: to retrieve feedback from potential users that is realistic and as
unbiased as possible. The basic principles that any item group should follow are
representativeness, relevance, diversity, clarity, simplicity and understandability
(Muñiz and Fonseca Pedrero 2009), each to a degree that suits the study.
Moreover, following previous experience (López-Rodríguez et al. 2012), the
questionnaire was hosted on the online platform LimeSurvey®, which allows
for different question formats. Thus we included several multiple-choice options
with comments, dropdown lists, lists with radio buttons, yes/no questions and text
questions (with short or long text), plus space for comments.
The survey included four sections. The introductory section (eight questions)
covered respondents’ personal and contact information. The introductory section
also included questions on nationality and professional experience in the field of
translation. In addition, we asked our respondents about their mother tongue and
working languages.
The first termino-lexicographic section (12 questions) explored the importance
and relevance that the design of different termino-lexicographic tools has for
translators in their everyday work. This section also included items eliciting data
on: the features of the reference tools respondents normally use for translation
purposes; which of these features are deciding factors when respondents choose
one tool over another; the type of tool and medium users preferred; and whether
they would want information to be displayed according to their level of expertise
(expert, semi-expert or layperson), as in EcoLexicon.
The second termino-lexicographic section (six questions) included items on
the usual sections of a ‘traditional’ lexicographic tool: user guides, symbols and
abbreviations, annexes, bibliography, information access, entry designs, etc.
Respondents were asked, for example, if these sections were relevant to them and
if they used them frequently. Our main interest here was the degree of ‘usability’
that these sections would have for our potential users – that is, ‘the measure of
how easily and effectively people can use something’ (Byrne 2006: 97). Usability
was considered a complex construct elicited by means of many proxy variables:
‘usefulness’, ‘frequency of use’, ‘relevance/importance’, ‘clear access/design/
presentation’, ‘difficulty/ease of use’, etc.
The third and last termino-lexicographic section of the survey (22 questions)
was designed to elicit information from our potential users about more fine-
grained issues, such as: what characteristics of a reference tool are the most (or
least) helpful when translating; what features are normally used and for what
type of problem; what strategies respondents used to retrieve more information
on a topic; and their points of view on polysemy and synonymy, politeness
and ideology, coherence between definitions, keywords in context and images
or illustrations. What we were ultimately interested in here was the degree of
impact that these specific features have on communicative and cognitive needs,
preferences, tasks and lacunae of our potential users.
Target population and administration of the questionnaire

Our population profile was deliberately heterogeneous, because we wished to get
a picture of the needs and interests of different types of translator as dictionary
user. Our respondents ultimately fell into three distinct categories: (a) specialized
freelance translators; (b) lecturers in specialized translation; and (c) university
students of specialized translation and terminology. The survey was administered
over three iterations during the 2012–13 academic year.
Before each iteration, we explained the purpose of the survey, as well as basic
lexicographic and terminographic concepts (making relevant distinctions and
introducing examples), so that respondents would be better able to understand
each item. These concepts included conceptual networks (Figure 5.1), concept
maps (Figure 5.2) and illustrations, which are an important part in the multimodal
interface of EcoLexicon and our prototype. Our aim was to elicit respondents’
opinions on the knowledge transmission potential of these items and whether or
not they should be included in our prototype.
The first two groups – that is, specialized freelance translators and lecturers
in specialized translation – answered from their own computers without the
researcher being present. These respondents were introduced to the survey by
means of a widely circulated email and via the front page of the survey, which
Salinización
Salinidad Polución
Ecosistema Calima
Cambio climático Ácido
Smog
Contaminación atmosférica
Atmósfera
Ácido sulfúrico
resultado de Lluvia_agua
compuesto de (material)
Contaminate del aire

Nimboestrato
Lluvia ácida Decloración
tipo de
Lluvia_proceso Compuesto quimico
Flora Altoestrato
resultado de causa
resultado de
Nutriente Eutrofización Precipitación_proceso
Dióxido de azufre
Lluvia residual
Masa de Acidificación
agua Lluvia engelante
Eutrófico
Proceso
Fauna Oxigeno
Azufre
Degradación del suelo
Figure 5.1 Conceptual network for the concept acid rain / lluvia áida, from EcoLexicon
included instructions and hyperlinks. Most respondents, however, filled out the
survey in one of several computer rooms at the University of Granada, University
of Córdoba, University of Málaga or Pablo de Olavide University (Seville) in
Spain, or the Aristotle University of Thessaloniki (Greece). These respondents
were specialized translation students from different backgrounds and levels:
from second- and third-year students following courses on terminology or
scientific and technical translation, to Master’s students on specialized translation
programmes. Both the researcher and the students’ lecturer for the respective
specialized translation course were present and available to answer any questions
related to the introductory tutorial to the survey or the survey itself. Non-remote
respondents had at least 90 minutes in which to complete the survey, as well as
internet connection, online and paper dictionary for consultation and projector
images. Remote respondents – that is, the freelance translators and lecturers, –
had no time or resource constraints. Those who were not already familiar with our
project or the questionnaire were appropriately introduced to both in our email.
Figure 5.2 Conceptual map for acid depositions / deposiciones áidas, in which acid rain is included, generated with CmapTools
Validation of the questionnaire
The questionnaire was validated in two ways. First, it was sent to 30 experts in
different areas – lexicographers, terminologists, specialized translators, professors
of translation – all of whom had doctorates from different universities in Europe.
Twenty experts responded.5
Experts were asked to validate the content of the questionnaire in terms of
its potential to measure the construct ‘translators’ and trainee translators’
expectations of dictionaries and encyclopaedias in any medium (paper, online,
other electronic) and of their main sections and characteristics’ (in the context of
the planning and implementation of a specialized lexicographic resource).
The cut-off point for the content validity index for each item was established at
0.70, following the recommendations of psychometrics expert José Manuel Romero
Sánchez. This cut-off point is usually applied in health sciences (Lynn 1986), an
area that has developed rigorous evaluation instruments (López-Rodríguez et al.
2012: 62) and one that shares with termino-lexicography the fact of being an applied
discipline that attempts to deliver a reliable service to society (García-Aragón 2016).
The overall content validity index for the questionnaire was very high, at 0.90375.
Only one item had to be removed because of a problem in its formulation.
Second, one semester after the administration of the questionnaire, respondents
were asked to participate in a retrospective validation of same. We asked our
original respondents nine questions6 about the questionnaire itself (as well as
other questions about MeteoTrad, which are described later in this chapter). The
nine questions contained statements about the survey contents, as well as the
structure of the survey, its length, the time it took to complete and how easy
it was to complete. Only 30 (14.93 per cent) of the original 201 respondents to
the questionnaire responded. However, the overall mean for each question was
extremely positive, at 4.214 on a scale of 1–5, in which 1 corresponded to ‘I
completely disagree’ and 5 to ‘I completely agree’ (with a positive statement). In
Figure 5.3, the standard deviation is also shown for each statement.
Figure 5.3 Statements and results of the user evaluation of the questionnaire
Survey results
Respondent profiles
Four out of 201 questionnaires were deemed unusable because they were
incomplete or unreadable owing to technical problems. We were therefore able
to analyse 197 surveys from 197 respondents with 48 items each, which amounts
to a potential total of 9,456 items. Because not all items were answered by all
respondents, however, the number of responses per item can vary.
All of the participants in our study had a higher education profile and 162
(82.23 per cent) of them had previous translation experience – although, for 145
respondents (73.6 per cent), this was non-professional translation experience.
Only 35 respondents (17.77 per cent) had no translation experience at all, whereas
16 respondents (8.12 per cent) reported both professional and non-professional
translation experience. The mean age was 22.33, 32 respondents (16.6 per cent)
being over the mean.
Most (153) respondents were Spanish (78.17 per cent of a total of 196
respondents who answered this question) and Greek nationals (9 respondents,
i.e. 4.6 per cent), the rest (34 respondents, 17.2 per cent) being from Argentina,
Austria, Belgium, Brazil, China, Estonia, France, Germany, Italy, Morocco, Peru,
Poland, the United Kingdom, the United States, Senegal and Syria. Their working
languages were (in order of frequency) Spanish, English, French, Modern Greek,
German, Italian, Russian and Arabic.
Main results
As previously mentioned, our survey results serve as a compass – that is, a guide
to help us to identify problems, solutions and patterns concerning our potential
users’ communicative and cognitive needs, preferences, tasks and lacunae when
translating. In this section, we present and interpret some of the most important
results, and discuss possible implications for our project.
The first termino-lexicographic section of the survey was meant to elicit
information on the importance of the design of different lexicographic tools for
translators, as well as on the features of the reference tools they normally use. We
wanted to find out about the degree of dependency of our target group on these tools
in their work, as well as their subjective evaluation of the reliability of these tools.
A key question was:
How would you rate the importance of dictionaries/encyclopaedias and other

related resources (terminological databases, forums, glossaries, etc.) for the
work of the translator?
1 no importance / 2 little importance / 3 some importance / 4 very important /

5 extremely important7
(Q.11 (1C) in the questionnaire, here in translation into English)
The mean score was 4.67, indicating that dictionaries are generally considered to
be ‘extremely important’.
In Q.12 (1D), concerning the frequency with which respondents use
termino-lexicographic resources, the mean was 4.5, again on a 1–5 scale, with
1 corresponding to ‘never’ and 5 indicating ‘always’. These results show that
translators value and nearly always use dictionaries in their work, so the content
and design of these resources certainly merits our attention.
In Q.30 (3C), to elicit information on what respondents used dictionaries for, we
provided a multiple-choice list of possible answers and asked respondents to rate the
frequency with which they used such resources on a scale of 1–5, 1 corresponding
to ‘never’ and 5 being ‘always’. The most frequently reported motivations were
‘to find an equivalent in another language’ (mean = 4.47) and ‘to find out what an
unknown term means’ (4.29), a finding that has motivated the inclusion of routes
1 and 2 in MeteoTrad. Other motivations, with mean scores ranging from 3.99 to
3.00, included the following (in descending order of frequency):
• to have a clearer idea of an obscure concept – that is, cognitive needs (3.89);
• to see a given word in context – that is, in concordances and example
sentences (3.5);
• to learn how a term is used grammatically and syntactically – that is, to
find information on associated prepositions, declensions, conjugations,
irregularities, plurals, etc. (3.49);
• to find synonyms and antonyms, including terminological variants (3.46);
• to find related fixed expressions – that is, phraseology (3.26);
• to find out how a term is used pragmatically – that is, its associated
connotations, register, stylistics, etc. (3.24);
• to check for the correct spelling – that is, orthography (3.22); and
• to check if a concept changes in a specialized field (3.01).
These results are in consonance with other answers from the third section of the
survey, which aimed to find out which characteristics of a termino-lexicographic
tool were the most (or least) helpful when translating. For instance, in Q.29
(3B), in descending order of frequency, when asked what most facilitates their
translation work when consulting dictionaries, 96 responses (15.2 per cent of the
total 631 responses) indicated ‘speed in finding what I need in an intuitive way’,
92 responses (14.55 per cent), ‘variety of terms and equivalents in context’, and
74 responses (11.74 per cent), ‘the interconnection of concepts or terms with one
another so that I get a clear picture of the domain’.
In Q.28 (3A), also in descending order of frequency, when asked what most
complicates their translation work when consulting dictionaries, 105 out of 643
responses (16.32 per cent) indicated ‘the lack of examples and specific cases’, 74
(11.52 per cent), ‘poorly organized information’, and 70 (10.8 per cent), ‘lack of
usage examples of a term’.8
In Q.13 (1E), respondents were also asked to name the ‘traditional’
lexicographic works (dictionaries, encyclopaedias, thesauruses, glossaries, etc.)
that they used in their everyday translation work. A fine-grained analysis of the
data provided allows us to summarize the most prominent characteristics of the
resources respondents mentioned as follows.
1 They are available online, so users have speedy access to the information they
need.
2 They are free or light versions, thus saving users money.
3 Most work with language pairs, displaying bilingual results from a
multilingual menu.
4 Most are non-specialized resources (i.e. they are general dictionaries) – this
answer is probably a result of the fact that respondents reported in Q.14
(1F) that they tend to rely on parallel texts, term bases and/or glossaries for
specialized consultations.
5 Most are backed by well-known publishing houses (Pons, Oxford, Larousse,
Collins) or prestigious institutions (Real Academia Española, the European
Union, universities).
6 Except for Wordreference.com, they do not have discussion forums,
respondents discussing problems with colleagues or native speakers instead.
In Q.15 (1G), in descending order of frequency, respondents indicated that

if they had to buy a dictionary, they would make their selection based on the
recommendation of their colleagues (128 out of the total 772 responses, i.e. 16.58
per cent), the quality of definitions – having the chance to check them first (123
responses, 15.9 per cent) and the subject field(s) of the dictionary (116 responses,
15 per cent). We could infer that if respondents were to have to spend money on a
useful tool, they would look for something that they do not usually have for free –
that is, for trustworthy specialized knowledge.
In Q.16 (1H), respondents were subsequently asked what type of dictionary
they would normally use for specialized translation. The results were clear: (a)
a bilingual specialized dictionary on a single subject field (38.9 per cent); (b) a
monolingual encyclopaedic dictionary on a single subject field (25.4 per cent);
and (c) a bilingual specialized dictionary on several related subject fields (17.1
per cent). For this item, the total number of answers was 379, because respondents
could choose more than one option for each of the variables number of languages,
type of lexicographic tool and number of subject fields. The most popular options
are apparent in Table 5.2.
In Q.37 (3J), the majority of our potential users (176 out of 188 respondents,
i.e. 93.6 per cent) valued the interdependence and consistency of the definitions of
related specialized concepts (worded as ‘coherence’) in a termino-lexicographical
tool.
In Q.19 (1K), when asked if they would like to see the knowledge in
dictionaries divided up according to whether it was aimed at experts, semi-experts
or laypeople, 138 out of 197 respondents (70 per cent) answered ‘yes’, with 37
respondents (18.8 per cent) answering ‘no’ and 22 respondents (11.2 per cent),
‘don’t know’ (Figure 5.4).
Table 5.2 Respondents’ preferences on number of languages, type of dictionary and
subject field
Number of languages
Bilingual 204 53.8 per cent

Monolingual 104 27.4 per cent
Multilingual 71 18.7 per cent
Type of reference work
Specialized dictionary 184 48.5 per cent

Encyclopaedic dictionary 118 31.1 per cent
Encyclopaedia 77 20.3 per cent
Subject fields
Single subject field 216 57 per cent

Several related subject fields 163 43 per cent
Note: n = 197
It is useful to divide the information in a dictionary

into experts, semi-experts and laypeople
11.2%
18.8%
70.0%
Yes No Don’t know
Figure 5.4 Answers for and against the division of knowledge into separate layers
for experts, semi-experts and laypeople
Note: n = 197
In Q.20 (1L), of those respondents who had answered the above question
affirmatively – that is, those who supported the division of knowledge according
to user expertise – 118 out of 192 respondents (61.5 per cent) voted for ‘a gradual
presentation of those levels in the same entry by means of a sign or symbol’ and
32 respondents (16.7 per cent) chose ‘a clear-cut distinction in the same entry by
means of a different section’. Other possible answers included a distinction by
means of annexes, different entries for experts, no distinction at all, etc.
As for the preferred medium for a specialized dictionary (Q.17, 1I), for example
paper or online, ‘online with a discussion forum’ was by far the most popular
choice (94 out of 197 respondents, i.e. 47.71 per cent), followed by ‘paper’ (38
respondents, 19.28 per cent) and ‘as an installed program’ (32 respondents, 16.5
per cent) (Figure 5.5).
Respondents were also asked about the importance they attribute to
illustrations in specialized contexts (Q.39 (3L)). We found out that images play
Preferred mediums
for a dictionary for specialized translators
3.67%
3.67%
5.20% 3.97
%
16.50% 47.71%
19.28%
online with a discussion forum paper

installed program CD/DVD
for e-book or tablet online without a discussion forum
PDF with external links
Figure 5.5 Preferred mediums in a specialized dictionary

Note: n = 197
an important role in translation for 136 out of 174 respondents (78.16 per cent) of
our respondents. In Q.40 (3M), they were asked about the frequency with which
they use illustrations as a reference. The mean score here was 3.33 (SD = 0.871)
on a scale of 1–5, 1 corresponding to ‘never’ and 5 being ‘always’. In Q.41 (3N),
users also showed a preference for ‘big and clear colour photographs’ (123 out of
197 respondents, i.e. 62.55 per cent) as the most suitable images from which to
properly acquire specialized information.
Respondents were also asked if dynamic conceptual networks, such as those
in EcoLexicon (see Figure 5.1), were useful resources in a specialized termino-
lexicographic work (Q.42 (3O)). The vast majority (87.4 per cent) answered
affirmatively. In answer to the next question on the role of such conceptual
networks (Q.43 (3P) – see Figure 5.6), most respondents indicated that they
would ‘complement (not substitute for) definitions of concepts’ (105 out of 197
respondents, i.e. 53.42 per cent). Results were similar in relation to the role of
Conceptual networks
in the dictionary could :
5.02% 3.66% 3.20%
34.70%
53.42%
substitute for a definition substitute for an explanation

complement a definition complement an explanation
none of the above
Figure 5.6 Choices on the possible roles of conceptual networks such as that for
acid rain in Figure 5.1
Note: n = 197
concept maps, such as that in Figure 5.2. Most of our respondents believed that
concept maps were complementary to the explanations and definitions provided
in termino-lexicographic tools (Q.44 (3Q) – see Figure 5.7).
When respondents were asked in Q.46 (3S) which of the above-mentioned
knowledge representation techniques (images, conceptual networks, concept
maps) was most appropriate and helpful for the acquisition of specialized
knowledge, a combination of the three, combined with definitions, was the most
popular choice (74 out of 197 respondents, i.e. 37.74 per cent), followed by
dynamic conceptual networks as found in EcoLexicon (51 respondents, 25.98
per cent) and concept maps (51 respondents, 25.98 per cent) (Figure 5.8).
All in all, this survey has shed some light on the use of termino-lexicographic
resources made by translators, as well as their preferences regarding the sort of
information included and how it is presented, the types of resource that they use
and the features. The results of the survey informed the design of a termino-
lexicographic prototype aimed at translators and called ‘MeteoTrad’.
Concept maps in the

dictionary could :
1.88%
7.04%
6.10%
43.66% 41.32%
substitute for a definition substitute for an explanation

complement a definition complement an explanation
none of the above
Figure 5.7 Choices on the possible roles of concept maps such as that for acid
depositions in Figure 5.2
Note: n = 197
In a dictionary, it would be easier for me to understand

and I like to find :
1.47%
8.83%
37.74%
25.98%
25.98%
networks, maps, images and definitions conceptual networks

concept maps schematic images
networks, maps and images
Figure 5.8 Choices on knowledge representation techniques

Note: n = 197
MeteoTrad: a dictiopedia on atmospheric

sciences for translators
MeteoTrad (García-Aragón 2014b) is a termino-lexicographic prototype that has
been designed according to the theoretical and methodological premises of termino-
lexicography (García-Aragón 2016) and the results of the survey described in
this chapter. MeteoTrad thus takes a hybrid approach that incorporates elements
of both the FTL, which focuses on user needs and lexicographic functions, and
FBT, which focuses on specialized knowledge acquisition. It is a user-oriented,
specialized termino-lexicographic project that aims to accommodate translators’
needs in a specific subject-field (atmospheric sciences, a subfield of EcoLexicon)
through the facilitation of the knowledge-acquisition process underlying the use
of specialized terms.
The prototype
In the design of the prototype, we tried to personalize translators’ access by allowing
them to choose different ‘routes’ depending on their lexicographic needs. The idea
behind the prototype is that when translators access MeteoTrad’s web interface,
they can choose a route for their query according to their priorities at a given point
in the translation process. For example, if a translator enters the query wet acid
deposition in MeteoTrad, he or she will decide whether he or she is searching for:
• its recommended equivalent(s) – that is, a communicative function from the

lexical point of view (route 1);
• its standard definition, thus a cognitive function (route 2);
• its usage – that is, a communicative function at the textual and discourse
level (route 3); or
• some standard knowledge about the concept wet acid deposition – that is,
a cognitive function to contextualize the concept within its domain (related
concepts and conceptual relations) (route 4) (Figure 5.9).9
Since the research process is not linear or clear-cut, when it comes to the results
shown on the screen of MeteoTrad for a particular route, translators have the
option of expanding the information related to that query, so that different routes
combine with different related information (see Figure 5.9).
Implementation of the prototype

The online implementation of the prototype is shown in Figure 5.10, in which the
four routes are displayed. When translators choose, for example, route 1 for the
query wet acid deposition from English into Spanish, they find one recommended
equivalent (deposición húmeda ácida) and other variants that can be used as
translation equivalents in Spanish (Figure 5.11). These variants are accompanied
by information on their use in Spanish.
Once the variants have been consulted, users can expand this information
by clicking on different dynamic options presented in order of relevance to their
previous query. Figure 5.12 shows an example of termino-lexicographically relevant
linking after a search in the ‘Equivalents’ route. By clicking on ‘Usage’, users can
get example sentences or phraseology; by clicking on ‘Definition’, they will retrieve
a definition of the term in English or Spanish; and finally, if they want to better
understand the relationship between this concept and other concepts, they can click
on ‘Knowledge’, and retrieve conceptual clouds from EcoLexicon and concept maps.
Although the idea is that MeteoTrad displays only information that is tailored
to a particular query (based on the user profile, directionality of translation and
type of query), the four routes are interconnected, as shown in Figure 5.12, so that
more information can be made available. In this way, if the translator performing
a search through route 1 wants to check for a standard definition (route 2), he or
she could choose to display a standard definition (for laypeople or semi-experts)
or an advanced definition (for experts) in English or Spanish. If translators want
to expand their knowledge on the concept wet acid deposition, they can click
on ‘Knowledge’ and then get instant access to the related concept network in
EcoLexicon (as in Figure 5.1) or to a concept map in the language in question.
1. EQUIVALENTS of [QUERY] from EN EN
into c. Knowledge of [QUERY] in EN or SP
i.e.: SP
SP
recommended equivalents, b. Definitions of [QUERY] in EN or SP
grammar, synonyms, GR GR
acronyms, translation notes a. Usage of [QUERY] in SP
2. USAGE of [QUERY] in EN
a. Equivalents of [QUERY] in SP or GR
show links to
i.e.: SP
example sentences, results results for route 1 b. Definitions of [QUERY] in EN, SP or GR
who can GR
filtered user queries grammar, parallel texts, query is processed in are shown in
according choose the most show links to
trad through
meteo [translators] collocations c. Knowledge of [QUERY] in EN, SP or GR
to appropriate route results for route 2
INTERNAL WEBSITE
trad meteo
3. DEFINITIONS of [QUERY] in EN
DATABASE which lead to INTERNAL results for route 3 show links to
a. Knowledge of [QUERY] in EN
the creation of i.e.: DATABASE
SP
multiple inheritance, WEBPAGE results for route 4 b. Definitions of [QUERY] in SP or GR
show links to
standard and advanced definitions, GR
individual user profiles
standard and advanced explanations, c. Knowledge of [QUERY] in SP or GR
(registering and logging in,
standard and advanced illustrations,
cookies, search history, etc.)
external links, videos a. Definitions of [QUERY] in EN, SP or GR d. Equivalents of [QUERY] in SP or GR
4. KNOWLEDGE about b. Knowledge of [QUERY] in SP or GR e. Usage of [QUERY] in EN

[QUERY] in EN
i.e.: conceptual networks, SP c. Equivalents of [QUERY] in SP or GR

concept maps,
etymo-cognitive maps, GR d. Usage of [QUERY] in EN
external links, videos
Figure 5.9 User query routes in MeteoTrad according to termino-lexicographic relevance for translators
Figure 5.10 Searching for wet acid deposition in MeteoTrad: the four access routes
Figure 5.11 Excerpt from the route ‘Equivalents’ for wet acid deposition in Spanish
Figure 5.12 Examples of termino-lexicographically relevant linking to related results in

other routes
Thus, in the design of MeteoTrad, we have tried to cater for the needs and
preferences of translators, as expressed by its potential users through the survey,
so that the tool can provide users with all necessary information for a quick
and easy translation process. Such information includes explicit indication of
directionality, as well as information on preferred terms, usage differences
between variants, schematic usage of the term, example sentences, definitions in
different languages, concept maps, etymo-cognitive maps (ECMs)10 for translators
working with Modern Greek (García-Aragón 2014a), etc.
User evaluation of MeteoTrad

After the implementation of the prototype of MeteoTrad, the 201 respondents to
the first survey were asked to evaluate its homepage and the different entries in
each route for the concept term wet acid deposition in different languages. As
with the evaluation of the survey itself, referred to earlier in this chapter, only 30
of the original respondents (14.93 per cent) replied. However, the responses were
extremely positive, with the grand mean for all questions reaching 4.465 on a
scale of 1–5, in which 1 represented ‘I completely disagree’ and 5, ‘I completely
agree’ (with a positive statement).
Figures 5.13–5.17 show the mean and the standard deviation for each statement,
organized by section, which sections are: ‘Homepage’, ‘Equivalents’, ‘Usage’,
‘Definitions’ and ‘Knowledge’.
In the Homepage section (Figure 5.13), there were eight statements with which
users had to agree or disagree, numbered as follows on the x-axis in Figure 5.13.
1 ‘The homepage is useful’

2 ‘The homepage is innovative’
3 ‘The homepage covers the needs of a translator’
4 ‘The homepage is well structured’
5 ‘The homepage is easy to use’
6 ‘The design of the homepage is intuitive’
7 ‘The design of the homepage is attractive’
8 ‘I would like to find this query system in similar resources’
Figure 5.13 Mean and standard deviation for each statement about the MeteoTrad
homepage
Note: 1 = completely disagree; 5 = agree completely
Figure 5.14 Mean and standard deviation for each statement about the Equivalents route
Figure 5.15 Mean and standard deviation for each statement about the Usage route
For all four routes (Equivalents, Usage, Definitions, Knowledge), users had to
agree or disagree with the following nine statements.
1 ‘The entry is useful’

2 ‘The entry is innovative’
3 ‘The entry is necessary for the task’
4 ‘The entry is well structured’
5 ‘The entry is easy to use’
6 ‘The entry lives up to my expectations’
7 ‘The entry makes my task easier’
8 ‘The contents of the entry are appropriate for the task’
9 ‘I would recommend this entry to colleagues with the same task’
Figure 5.16 Mean and standard deviation for each statement about the Definitions route
Figure 5.17 Mean and standard deviation for each statement about the Knowledge route
The general mean for all statements about every route was extremely high: 4.465
on a scale of 5. This means that users were extremely satisfied with the different
aspects of each route and with MeteoTrad in general.
Conclusions
Even though translation studies, terminology and lexicography have evolved
independently, they converge in user-centred, functionalist theories of
communication and knowledge transfer – an approach that can be applied to
termino-lexicographic resources for translators. In this chapter, we have described
MeteoTrad, an incipient termino-lexicographic resource for translators. Although
MeteoTrad is partly based on EcoLexicon – mainly on its frame-like structures and
specialized corpus data – MeteoTrad has been designed from scratch from a new
general, unitary theory of terminology and lexicography (termino-lexicography),
by integrating both FBT and the FTL in an interdisciplinary construct in which user
tasks, lacunae, preferences, cognitive processes and language play a crucial role
(García-Aragón 2016). Following these principles, MeteoTrad aims specifically
to help translators to translate texts in atmospheric sciences to and from English,
Spanish or Modern Greek.
An extensive survey with 201 respondents prior to the design of MeteoTrad
shed some light on broad design issues, on what should be included or excluded,
and on the termino-lexicographic needs and preferences that have to be met when
translators perform translation tasks in specialized contexts. Despite claims that
users may not know what they want or what is best for them, and some scepticism
about self-reported data (see, e.g., Nielsen and Levy 1994), these results show, at
least, a ‘north’ to which lexicographers and terminographers should set their sails:
the problems, expectations and preferences of translators. Their answers have
formed the basis for the design of MeteoTrad, a tool ‘by translators for translators’,
the prototype for which has the merit of having been evaluated in a ‘concrete
instantiation’ (Nielsen and Levy 1994: 75). We have also seen that the results
of the survey are consonant with our previous assumptions about respondents’
communicative and cognitive needs at the conceptual, textual, lexical and subject-
field levels, as well as with different translation sub-competences identified in the
translation studies literature.
Inspired by the FTL, we have individualized these needs into specific query-
functions, or routes, implemented in MeteoTrad and have presented their potential
combinations. Moreover, based on the premises of FBT, the design of MeteoTrad
has tried to facilitate the knowledge-acquisition process associated with the
use of specialized terms. We hope that MeteoTrad will, in time, become a free,
online specialized dictionary on atmospheric sciences that will help translators to
translate to and from several languages using a bilingual interface, with discussion
forums, backed up by university research groups and other features from which
our users will be able to choose.
In short, both the results of our survey on translators’ needs, preferences and
lacunae, and the design of MeteoTrad, have given us the chance to study the
reference needs of translators, their preferences in relation to traditional and new
termino-lexicographic resources, the way in which they interact with them, and
their focus on conceptual, linguistic and visual information.
Acknowledgements
This research was carried out within the framework of project FF2014-52740-P,
Cognitive and Neurological Bases for Terminology-Enhanced Translation
(CONTENT), and CombiMed, Combinatory Lexis in Medicine: Cognition, Text
and Context (FFI2014-51899-R), funded by the Spanish Ministry of Economy and
Competitiveness.
Notes
1 Since not many resources are specially designed for translators, some of these
professionals resort to creating their own terminological resources from ad hoc corpora
or from their own translation memories (Durán Muñoz 2010: 55–6).
2 Where they do exist, such resources are usually called ‘dictionaries for translators’ or
‘translation dictionaries’ (Tarp 2013, 2014).
3 For more information on EcoLexicon, see http://ecolexicon.ugr.es/en/aboutecolexicon.
htm [accessed 12 March 2016].
4 The survey can be accessed online at http://tinyurl.com/os4uavr and a pdf version of
the questionnaire is available at http://tinyurl.com/ngqw5sq [accessed 12 March 2016].
5 In alphabetical order: Julian Bourne; Miriam Buendía-Castro; Narciso Contreras-
Izquierdo; Pamela Faber; Vicente Fernández-González; Pedro Fuertes-Olivera;
Joaquín García-Palacios; Pilar León Araúz; Clara Inés López-Rodríguez; Jadwiga
Linde; Nava Maroto; José Mateo; Ricardo Muñoz-Martín; Konstantinos Paleologos;
Juan Antonio Prieto-Velasco; Arianne Reimerink; Bryan Robinson; Beatriz Sánchez-
Cárdenas; Miguel Sánchez-Ibáñez; and José Manuel Ureña-Gómez-Moreno.
6 The nine questions can be accessed online at http://tinyurl.com/n245x9f, under the
heading ‘sobre el cuestionario que cumplimentaste’.
7 The original question in Spanish was as follows:
¿Qué importancia tiene para ti el diccionario/enciclopedia y otros recursos afines

(bases terminológicas, foros, glosarios, etc.) para la labor de un traductor?
1 ninguna / 2 poca / 3 alguna / 4 mucha / 5 muchísima
In the following examples, we provide an English paraphrase of the original Spanish

question.
8 The number of responses here exceeds the number of respondents, because respondents
could select several of the 13 options specified or introduce a further answer in a field
labelled ‘other’.
9 Note here that users must indicate the language in which the query is made (English,
Spanish or Modern Greek are the options currently available) and the target language,
in the case of route 1.
10 Etymo-cognitive maps are infographic maps in which concept–term relations can be
logically structured in word clusters of etymologically related Greek words:
ECMs thus become representations of the individual’s specific lexical structuring

and understanding of a language, and are based on ad hoc categorizations and
motivations. These maps are always heterarchical, and can be mono-, bi-, or
multilingual depending on the learner’s background and objectives.
(García-Aragón 2014a: 40)
References
Balliu, C. 2013. ‘El legado de Eugene Nida en la traductología francófona’. In E. Ortega
Arjonilla (ed.). Translating Culture; Traduire la Culture; Traducir la Cultura. Granada:
Comares, 91–6.
Bergenholtz, H., and S. Tarp. 2003. ‘Two Opposing Theories: On H. E. Wiegand’s Recent
Discovery of Lexicographic Functions’. Hermes 31: 171–96.
Bergenholtz, H., and S. Tarp. 2004. ‘The Concept of Dictionary Usage’. Nordic Journal of
English Studies 3(1): 23–36.
Bergenholtz, H., and S. Tarp. 2010. ‘Lexicography or Terminography? The Lexicographer’s
Point of View’. In P. A. Fuertes Olivera (ed.). Specialized Dictionaries for Learners.
Berlin/New York: Mouton de Gruyter, 27–36.
Bergenholtz, H., S. Nielsen and S. Tarp (eds). 2009. Lexicography at a Crossroads:
Dictionaries and Encyclopaedias Today, Lexicographical Tools Tomorrow. Bern:
Peter Lang.
Biau Gil, J. R., and A. Pym. 2006. ‘Technology and Translation: A Pedagogical Overview’.
In A. Pym, A. Perestrenko and B. Starink (eds). Translation Technology and Its
Teaching. Tarragona: Intercultural Studies Group, 5–19.
Byrne, J. 2006. Technical Translation: Usability Strategies for Translating Technical
Documentation. Dordrecht: Springer.
Byrne, J. 2012. Scientific and Technical Translation Explained. Manchester: St Jerome.
Candel Mora, M. Á. 2010. ‘Las memorias de traducción para la enseñanza de la traducción
especializada’. In E. Alarcón Navío (ed.). La traducción en contextos especializados:
Propuestas didácticas. Granada: Atrio, 269–76.
Downing, S. M. 2006. ‘Twelve Steps for Effective Test Development’. In S. M. Downing
and T. M. Haladyna (eds). Handbook of Test Development. Mahwah, NJ: Lawrence
Erlbaum Associates, 3–25.
Durán Muñoz, I. 2010. ‘Specialized Lexicographical Resources: A Survey of Translators’
Needs’. In Proceedings of ELEX2009: eLexicography in the 21st Century – New
Challenges, New Applications. Louvain-la-Neuve: Presses Universitaires de Louvain,
55–66.
Durán Muñoz, I. 2011. ‘Recursos electrónicos para la búsqueda terminológica en
traducción: clasificación y ejemplos’. Revista Tradumàtica: tecnologies de la traducció
9: 137–46.
Duvå, G., A.-L. Laursen and L. Maidahl. 1992. ‘Brugerundersøgelser vedrørende
oversættelse af fagtekster’. In R. V. Fjeld (ed.). Nordiske studier i leksikografi: rapport
fra Konferanse om leksikografi i Norden. Oslo: Nordisk forening for leksikografi,
105–33.
Faber, P. (ed.). 2012. A Cognitive Linguistics View of Terminology and Specialized
Language. Berlin/Boston: Mouton de Gruyter.
Faber, P., and C. I. López-Rodríguez. 2012. ‘Terminology and Specialized Language’. In
P. Faber (ed.). A Cognitive Linguistics View of Terminology and Specialized Language.
Berlin/Boston: Mouton de Gruyter, 9–31.
Faber, P., P. León Araúz and A. Reimerink. 2011. ‘Knowledge Representation in
EcoLexicon’. In Proceedings of TISLID’10: Technological Innovation in the Teaching
and Processing of LSPs. Madrid: Universidad Nacional de Educación a Distancia,
367–86.
Franco Aixelá, J. 2001. ‘Prescriptivismo y descriptivismo: objetivos de una teoría de la
traducción’. In P. Y. Raccah and B. Sáiz Noeda (eds). Lenguas, Literatura y Traducción.
Madrid: Arrecife, 157–73.
Fuertes-Olivera, P. A., and S. Tarp. 2014. Theory and Practice of Specialised Online
Dictionaries: Lexicography versus Terminography. Berlin: Mouton de Gruyter.
García-Aragón, A. 2014a. ‘An Etymo-Cognitive Approach to Modern Greek Vocabulary
Learning’. In N. Lavidas, T. Alexiou and A. M. Sougari (eds). Major Trends in
Theoretical and Applied Linguistics, Vol. 3. London: Versita, 33–50.
García-Aragón, A. 2014b. MeteoTrad: Meteorology and Climatology for Translators.
Creative Commons (2014–2016). Available at: http://www.meteotrad.com [accessed
12 March 2016].
García-Aragón, A. 2016. Lexicografía especializada para traductores: una propuesta
multidisciplinar para el diseño de diccionarios a medida del usuario (inglés-español-
griego). PhD thesis. Universidad de Granada.
Hatim, B., and I. Mason. 1995. Teoría de la traducción: una aproximación al discurso.
Barcelona: Ariel.
Hurtado Albir, A. 2007. Traducción y traductología: introducción a la traductología.
Madrid: Cátedra.
León Araúz, P., P. Faber and C. Pérez Hernández. 2008. ‘LSP Dictionaries and Their
Genuine Purpose: A Frame-Based Example from MARCOCOSTA’. In Proceedings
of the XIII EURALEX International Congress. Barcelona: Institut Universitari de
Lingüística Aplicada (Universitat Pompeu Fabra)/Documenta Universitaria, 997–1008.
León Araúz, P., A. Reimerink and A. García-Aragón. 2010. ‘Context-Based Modelling
of Specialized Knowledge’. Information Technologies and Knowledge 4(2): 122–42.
León Araúz, P., A. Reimerink and A. García-Aragón. 2013. ‘Dynamism in Context in
Specialized Knowledge’. Terminology 19(1): 31–61.
LexiCon Research Group. n.d. EcoLexicon: Terminological Knowledge Base. University
of Granada. Available at: http://ecolexicon.ugr.es/ [accessed 12 March 2016].
López-Rodríguez, C. I., M. Buendía-Castro and A. García-Aragón. 2012. ‘User Needs to
the Test: Evaluating a Terminological Knowledge Base on the Environment by Trainee
Translators’. Journal of Specialised Translation 18: 57–76.
Lvóvskaya, Z. 1997. Problemas actuales de la traducción. Granada: Método Ediciones.
Lynn, M. R. 1986. ‘Determination and Quantification of Content Validity’. Nursing
Research 35(6): 382–6.
Moya, V. 2004. La selva de la traducción: teorías traductológicas contemporáneas.
Madrid: Cátedra.
Muñiz, J., and E. Fonseca Pedrero. 2009. Construcción de instrumentos de medida en
psicología. 6th edn. Universidad de Oviedo: Formación continuada a distancia, Consejo
General de Colegios Oficiales de Psicólogos.
Nida, E. A., and C. Taber. 1986. La traducción: teoría y práctica. Madrid: Cristiandad.
Nielsen, J., and J. Levy. 1994. ‘Measuring Usability: Preference vs Performance’.
Communications of the ACM 37(4): 66–75.
Nielsen, S. 2008. ‘The Effect of Lexicographical Information Costs on Dictionary Making
and Use’. Lexikos 18: 170–89.
Nielsen, S. 2010. ‘Specialized Translation Dictionaries for Learners’. In P. A. Fuertes-
Olivera (ed.). Specialized Dictionaries for Learners. Berlin/New York: Mouton de
Gruyter, 69–82.
Nord, C. 1997. Translating as a Purposeful Activity: Functionalist Approaches Explained.
Manchester: St Jerome.
Reiss, K., and H. Vermeer. 1984. Grundlegung einer allgemeinen Translationstheorie.
Tübingen: Niemeyer.
Roberts, R. P. 1997. ‘Using Dictionaries Efficiently’. Paper presented at the 38th Annual
Conference of the American Translators Association, San Francisco, CA, 5–8
November. Available at: http://www.dico.uottawa.ca/articles-en.htm [accessed 12
March 2016].
Sánchez Ramos, M. M. 2004. El uso de diccionarios electrónicos y otros recursos de
Internet como herramientas para la formación del traductor (inglés-español). PhD
thesis. Universitat Jaume I, Castellón.
Seleskovitch, D., and M. Lederer. 1984. Interpréter pour traduire. Paris: Didier
Érudition.
Strandvik, I. 2013. ‘Barreras culturales y traducción institucional: el caso de la Unión
Europea’. In E. Ortega Arjonilla (ed.). Translating Culture, Traduire la Culture,
Traducir la Cultura. Granada: Comares, 325–31.
Tarp, S. 2005. ‘The Pedagogical Dimension of the Well-Conceived Specialised Dictionary’.
Iberica 10: 7–21.
Tarp, S. 2008. Lexicography in the Borderland between Knowledge and Non-Knowledge:
General Lexicographical Theory with Particular Focus on Learner’s Lexicography.
Tübingen: Max Niemeyer Verlag.
Tarp, S. 2012. ‘Do We Need a (New) Theory of Lexicography?’ Lexikos 22: 321–32.
Tarp, S. 2013. ‘What Should We Demand from an Online Dictionary for Specialized
Translation?’ Lexicographica: International Annual for Lexicography 29(1): 146–62.
Tarp, S. 2014. ‘Reflexiones sobre el papel y diseño de los diccionarios de traducción’.
MonTi 6: 61–89.
Tercedor Sánchez, M., C. I. López-Rodríguez and P. Faber. 2013. ‘Working with Words:
Research Methodologies in Translation-Oriented Lexicographic Practice’. TTR:
Traduction, terminologie, rédaction 25(1): 181–214.
Toury, G. 1980. In Search of a Theory of Translation. Tel Aviv: The Porter Institute for
Poetics and Semiotics.
Varantola, K. 1998. ‘Translators and Their Use of Dictionaries’. In B. T. S. Atkins (ed.).
Using Dictionaries. Tübingen: Niemeyer, 179–92.
Vermeer, H. J. 1987 ‘What Does It Mean to Translate?’ Indian Journal of Applied
Linguistics 13(2): 25–33.
6 Assessing user interface needs of
post-editors of machine translation
Joss Moorkens and Sharon O’Brien
Introduction
Translation memory (TM) and machine translation (MT) were, until quite
recently, considered to be distinct and diverging technologies. The current trend
in translation technology, however, is to attempt to create a synergy of the two.
At present, the TM tools used for the last decades to recycle human translation are
being adopted also for the task of post-editing MT output. We consider that while
these existing translation editor interfaces are by now familiar and functional for
translators working with TM, any support for post-editing or integration with
MT has tended to be appended as an afterthought. Post-editing of MT output is
different from revision of an existing translation suggested by a TM – or, indeed,
from translation without any suggestion whatsoever – primarily because the
types of revision differ. Machine translation output tends to include mistakes that
professional human translators would not generally make. When this is coupled
with the fact that few professional translators have received training either in MT
technology or in post-editing practices to date, the result is often apprehension
among translators with regard to the post-editing task, along with a high level of
frustration. Some of the most common complaints from translators about the task
of post-editing stem from the fact that it is an edit-intensive mechanical task that
requires correction of basic linguistic errors over and over again (Guerberof 2013;
Moorkens and O’Brien 2014). Understandably, translators see this task as boring
and demeaning, and especially despise it when the ‘machine’ does not ‘learn’
from its mistakes or from translators’ edits. Kelly (2014) even goes so far as to
call this task ‘linguistic janitorial work’.
This chapter describes our first steps in an ongoing effort towards creating
specifications for user interfaces (UIs) that better support the post-editing task,
with a view to making the task less boring, repetitive and edit-intensive for the
human translator. While our focus is on features for post-editing, our results
demonstrate that, according to translator-users, even the human translation task
is not well supported by existing tools. We therefore also discuss basic features
of TM tools, as well as those features that are used to facilitate MT post-editing
using TM interfaces.
The project began with a pre-design survey of professional translators,
focusing on features of the translation UI that they commonly use. The survey
110 Joss Moorkens and Sharon O’Brien
was followed by a series of interviews with professional translators – all of whom
have experience with post-editing – to examine their survey responses in more
detail and to discuss the specific challenges of post-editing, thus bringing more
potential user insight to the design process.1 We assume from the outset that a
stand-alone editor for post-editing is not required, but features and functionality
as specified could instead be built into existing translation editing environments
to better integrate with MT systems and to support the post-editing task.
In this chapter, we first define ‘post-editing’ and present previous relevant research
on interfaces used in post-editing. We explain how software designers are currently
responding to the emergent integration of TM and MT. We go on to describe our
survey and interview design, and to present a summary of the results of the survey,
followed by the interview findings. For reasons of space, the final UI specifications
will be published elsewhere. Our conclusions are presented in the final section.
Interfaces for editing human translation and

post-editing machine translation
Somers (1998: 138) describes post-editing of MT as ‘tidying up the raw output,
correcting mistakes, revisiting entire, or, in the worst case, retranslating entire
sections’ of a machine-translated text. This task can be further subdivided into ‘light’
(or ‘fast’) post-editing, to create output that is understandable and readable, but not
necessarily stylistically perfect, and ‘full’ (or ‘conventional’) post-editing, with the
aim of producing quality equivalent to ‘that of a text that has been translated by a
human linguist’ (de Almeida 2013: 13). The type of post-editing chosen depends on
the purpose of the translated text and the financial resources available.
In professional translation workflows, once the source text has been machine
translated, the output is presented to the translator-user using a suitable interface. It
is increasingly common for this interface to be one and the same as that provided by
the TM tool. The TM UI is where specialized translators translate from scratch, or
edit legacy translations when the source text segment is the same as or similar to one
that has been translated previously. Target text segments from the TM are assigned
a match percentage based on the difference between the source text to be translated
and the source text in the TM (a match of less than 100 per cent is known as a ‘fuzzy
match’). This gives the translator an estimate of the amount of similarity with a
source segment previously stored in the TM. The new translation or edited fuzzy
match is then saved to the TM, dynamically improving future match suggestions. It
is increasingly common for a TM UI to have a facility to use MT when no TM target
text match is available or when the TM match’s fuzzy match percentage is low.
There is no universally accepted threshold above which MT output is considered to
require less editing effort than a TM fuzzy match, but research has suggested that
raw MT editing effort may be equivalent to that required for 80–90 per cent fuzzy
matches (O’Brien 2006; Guerberof 2008), although this would depend on the text
type, language pair and the raw quality from the MT engine.
Given the growing importance of TM interfaces in the post-editing process,
it is worthwhile considering the extent to which they meet their users’ basic
Assessing user interface needs 111
translation editing needs, even before post-editing requirements are factored in.
Previous research suggests, unfortunately, that user-centred design (UCD) has not
been general practice in TM technology. Surveying TM UIs in 2006, Lagoudaki
(2008: 17) found that industry research and development was mostly motivated
by ‘technical improvement of the TM system and not how the TM system can
best meet the needs of its users’. She added that ‘systems’ usability and end-users’
demands seem to have been of only subordinate interest’ in TM system development
(Lagoudaki 2008: 17). In practice, TM users are usually ‘invited to provide feedback
on an almost finished product with limited possibilities for changes’ (Lagoudaki
2006: 1). Lagoudaki concluded that TM tool users wanted simplicity in their UI, not
necessarily meaning fewer features, but focusing on stable performance, improved
interoperability and high compatibility between file formats.
The non-UCD design process goes some way towards explaining why Lagoudaki
found that users were widely dissatisfied with their translation editing interface,
despite (at that time) 14 years of TM tool development. This dissatisfaction was
reiterated in McBride (2009: 125), in which one forum contributor is reported
as saying that the user was not the focus of the design process, because tool
developers hope ‘above all to sell to giant corporations, who will put pressure
on translation agencies to buy, who will likewise pressure translators to buy’.
According to Ozcelik and colleagues (2011: 7), end-user involvement in software
development is frequently mitigated, because ‘often the person who decides on
purchase is not really the end user’.
Other research has looked at the types of post-edit that are typically made by
human post-editors. De Almeida (2013), for example, found that post-edits (in
English to French and English to Brazilian Portuguese) typically include changes
such as word reordering, addition or removal of capitalization and changes to the
gender or number of a word. A post-editor may find himself or herself repeating
edits throughout a project, for example changing the same word from masculine
to feminine inflection every time it occurs, with no associated improvement to the
MT suggestions (De Almeida 2013). Koponen (2012) reported similar edits in
English-to-Spanish post-editing, noting that word-order changes were perceived
by her study participants as being more cognitively challenging than correction
of an individual word.
Unfortunately, current TM UIs are usually incapable of providing the post-
editor with an estimate of editing effort required for each segment or of assisting
with these common edits, although several research projects are now under way
that aim to create new enhanced computer-assisted translation (CAT) tools, by
adding functionality to assist post-editing. An early example of such a project,
TransType, integrated interactive machine translation (IMT), with varying levels of
success, to suggest completions of a segment that the translator had already started
to translate (Langlais et al. 2002). The technique used was similar to that used
in predictive texting. Later projects integrated functions that have only recently
become feasible, such as the use of translation quality estimation to recommend
TM or MT matches (as proposed by Dara et al. 2013), and most are not in use by
professional translators or post-editors at the time of writing. Another tool under
active research and development, iOmegaT, is a version of popular open-source
software (OSS) CAT tool Omega-T that retains information on edits carried out by
translators for post-translation analysis. This information can give valuable detail
to researchers and managers of post-editing activities, but the UI itself offers no
novel functionality for post-editors (Moran and Lewis 2011). The Matecat project,
meanwhile, was an industry–academia collaboration (since commercialized) that
aimed to create a web-based CAT tool to include estimation of MT quality and
incremental ‘tuning’ of the MT output based on post-edits (Cettolo et al. 2013).
At the time of writing, it is in full production use by the project’s industry partner,
although MT quality estimation has not yet been incorporated (de Souza et al.
2014). The associated Casmacat project focused further on novel functionality
deployed in a web-based platform, adding interactive MT prediction, intelligent
auto-completion, word alignment visualizations and confidence measures.2
Casmacat also added integration with eye-trackers and e-pens, and has subsequently
been made available as OSS for end-users (Koehn et al. 2015). Two other tools,
PET (Aziz et al. 2012) and Caitra (Koehn 2009), were developed for post-editing
research purposes, although they are not actually used in production. Notably,
prior to the current research, there has not been a focus on what functionality users
would like to see in a tool for post-editing, because the MT research community
has had a tendency to ‘focus on system development and evaluation’ rather than to
consider end-users (Doherty and O’Brien 2013: 4). Our work builds on previous
research by gathering several possibilities for a post-editing UI, and inviting user
input into whether and how this may be implemented.
Research design
The main objective in this research is to create user-focused specifications
for editing interfaces to better support the post-editing task. Our two research
questions are as follows.
1 Can we get pre-design information from users to redress the balance of user/
engineering input that is common in translation tool development?
2 What are the ‘pain points’ in post-editing and how can these be addressed in
a translation tool?
The method employed in answering these questions was a pre-design user survey
(Tidwell 2006), followed by detailed interviews with several of the survey
participants. The findings from this initial research may form a starting point for
tool development, which should involve evaluation and validation (or otherwise)
of the specifications as gathered from direct observation of users.
Survey
Our pre-design survey had five broad sections focusing on (a) respondents’
biographical details, (b) current working methods, (c) concepts of the ideal UI,
(d) presentation of TM matches and MT output, and (e) intelligent functions to
combine TM and MT matches. The survey contained ideas for specific features
that we considered might serve post-editors, based on common edits reported in
research and post-editing functions currently in development within the research
community (listed earlier in this chapter). Respondents to the survey were also
able to give more detailed comments or suggestions immediately following
specific questions.
The survey was carried out via the Internet using the LimeService platform
(http://www.limeservice.com), and required completion of an ‘informed
consent’ section prior to beginning the main body of the survey. In the first
section of the survey, participants were asked about their length of experience
as a translator and as a post-editor, in years and months. They were asked about
their professional status (freelance or employed by a company), and their views
on TM and MT technology respectively (‘I like using it’; ‘I dislike using it’; ‘I
use it because I have to’; ‘I use it because it helps in my work’; ‘MT is now an
advanced technology’; ‘MT is still problematic’).
In the second section, participants were asked for their source and target
languages, what editing environments they currently use and what they like
most about their current tools, and to ‘describe one aspect of your current editing
environment that frustrates you’. These were all to be answered in free text. They
were asked whether they customize their translation UI and, if so, what elements
they customize. Finally, they were asked about preferred screen layouts for source
or target texts and for glossaries.
The third section focused on those features that respondents would like to see
in the post-editing environment that are ‘not currently available in (their) regular
translation editing environments’ – again, leaving a free-text box for response.
Following questions about keyboard shortcuts and whether respondents preferred
a simple or rich-featured UI were a series of questions about specific types of post-
edit that respondents might like a keyboard shortcut to automate, with answers to
be chosen on a four-point Likert scale (see Figure 6.1), and a query about whether
a macro-creation tool would be useful.
Questions in the fourth section addressed the combination of TM features with
support features for MT and post-editing.
Finally, participants were asked whether they would leave an email address for
further participation in an interview, noting that, in doing so, they would waive
their anonymity. The survey went live on 7 May 2013 and a link was sent to six
localization companies, which disseminated it internally (see ‘Acknowledgments’
at the end of this chapter). The survey was closed at the end of business hours on
6 June 2013.
Interviews
The follow-on interviews were largely based on the survey results. These interviews
also presented an opportunity to see whether participants had requirements for
a post-editing UI that had not been identified in the survey responses. The final
Figure 6.1 Survey question with radio button responses
question was an open one: ‘Do you have any other suggestions on how to support
post-editing through the UI?’ Interviews took place between 2 July and 2 August
2013 via the Skype ‘voice over Internet protocol’ (VoIP) tool, and were recorded
using Callnote.
Survey results
Demographics
The survey had 403 respondents, 231 of whom answered all sections.3 Given
its method of dissemination, responses were somewhat biased by information
technology (IT) localization practices. At the same time, this sector has, in recent
years, embraced MT, and we therefore expected to access translators who had
acquired post-editing experience and who would thus make informed respondents.
Of the respondents, 280 completed the biographical section. Figure 6.2 shows the
age ranges of these participants.
Most participants reported that they had 6–12 years’ translation experience,
while 26 participants claimed more than 20 years’ experience. Reported post-
editing experience was mostly between one and three years, with 69 participants
reporting no experience of post-editing. All but three of the 42 respondents aged
20–30 had some experience of post-editing (at most two years). Roughly 80 per
cent of respondents aged between 31 and 50 (125 respondents) had experience
of post-editing (usually between two and six years), and just over half (17
respondents) aged over 50 had post-editing experience.
Of the total number of respondents, 29 per cent (81) reported that they work
as freelancers without an agency, 31 per cent (85), that they work closely with
one agency on a freelance basis (nine participants work on a freelance basis with
several agencies), and 23 per cent (63) were translation or localization company
employees. A further 21 respondents run their own companies. This cohort
Figure 6.2 Participants’ age range

Note: n = 280
represents a good spread of work profiles typical of the translation industry.

A statistically significant association was found between translators’ age and
professional status: respondents under the age of 30 are more likely to be company
employees (67 per cent, or 23), whereas those over 30 are more likely to work
on a freelance basis (71 per cent, or 148). The proportion employed directly by a
company drops to 26 per cent (23) for those aged 30–40, falling to 6 per cent (2)
for those aged over 50.
Those respondents who reported that they like using TM technology comprised
56 per cent (153), as compared with 18 per cent (49) who said that they like using
MT. Some 75 per cent of participants (206) report using TM because it helps with
their work, whereas 30 per cent (83) report using MT for the same reason. Just
over half hold the view that MT is ‘still a problematic technology’ (56 per cent,
or 149). Fewer respondents aged over 40 agreed that ‘MT was still problematic’,
which suggests that they do not feel threatened by MT; taken in conjunction with
the older group’s lesser post-editing experience, however, it could also mean that
they have less familiarity with MT and its associated errors. These differences
aside, responses were consistent between age ranges.
Of 280 participants, 223 (80 per cent) translate from English – a reflection
of the nature of the respondents and the companies who promoted the survey –
although many translate from more than one source language.4 Target languages
are reasonably well spread among participants (Table 6.1). This spread of target
languages was important for the survey results, because the post-editing task can
vary depending on the target language in question and its typical linguistic features.
Table 6.1 Participants’ target languages
Target language No. of participants
Arabic 5
Chinese 24
Czech 11
Danish 3
Dutch 7
English 49
Finnish 4
French 34
German 26
Greek 5
Hindi 3
Hungarian 5
Italian 18
Japanese 17
Korean 4
Malay 1
Norwegian 3
Polish 1
Portuguese 24
Russian 7
Spanish 27
Swedish 6
Thai 3
Turkish 4
Urdu 3
Current editing environment

Of the 246 participants who provided details of their current editing environments,
63 per cent (155) use more than one environment regularly. Of these, 74 per cent
(182) use a version of the SDL Trados TM tool. Company employees are more
likely to use SDL Trados; 109 of 152 freelance translators (72 per cent) and 65 of
76 (84 per cent) company employees said that they use a version of SDL Trados.5
Among company employees surveyed, 62 per cent (39) said that they use multiple
tools, but the rate was even higher among freelancers (68 per cent, or 116).6
Contrary to our expectations, 38 per cent of participants (94) use Microsoft Word
for post-editing, which suggests that MT and TM are not currently as integrated
as we had thought. Figure 6.3 shows the number of users per editing UI among
survey participants. Some other tools used by fewer than 15 participants were
XTM (13), Alchemy Catalyst (12), OmegaT (8), Star Transit (5), TransStudio (5)
and Alchemy Publisher (1). Twenty-eight participants also listed proprietary tools
(Translation Workspace, Helium and MS LocStudio).
Roughly half of the participants in this survey section reported unhappiness
with the default layout, colouring and display of mark-up tags in their current
editing UI. Fifteen complained specifically about poor layout or visibility, outdated
UIs and too many product updates. ‘The UI is not user friendly,’ wrote one,
‘each UI uses their own different shortcuts, there is an inability to see segments
comfortably.’ Seven participants mentioned compatibility issues and problems
with tags. Moreover, 66 per cent (167) of participants would rather customize
their editor than use the default set-up; 79 per cent of those 167 respondents adjust
their onscreen layout, 74 per cent adjust tag visibility, 68 per cent adjust font type
and 23 per cent adjust colours.
Performance issues figured strongly among survey comments, with 19
participants complaining about bugs, errors and slow response times within their
current UI. One participant wrote: ‘I work mainly in [software name], which is
useful [but] an incredibly fragile piece of software that has caused me to lose
time due to crashing or failing to save output files correctly.’ Twenty-one other
participants stated that they have experienced formatting problems.
Twenty-five participants expressed unhappiness with the quality of MT output
and MT support within their current tool. One wrote that ‘sometimes the quality
of MT makes me stay longer at a (translation unit) than I would having no MT
to deal with’. This was a recurring bugbear in open responses throughout the
survey. Other problems mentioned were the high learning curve with CAT tools.
On the other hand, 30 participants said that they are happy with their current UI,
although not necessarily in the most positive terms: ‘I’m so used to it, that I can’t
find anything frustrating.’
When asked what they liked most about their current tools, many (33)
mentioned performance, ease of use and stability. Seventeen mentioned specific
features such as auto-propagation, integrated quality assurance (QA) checking
and concordance searches. Six participants wrote that they liked their current UI,
with one writing ‘the editing changes are clearly marked and text before and after
are displayed side by side’.
UI wish list
The importance of customizability was emphasized in many of the 245 responses
to this section of the survey. Of participants, 63 per cent (152) expressed a
preference for a customizable UI and 57 per cent (138), a clean and uncluttered
UI. In response to a question about features currently unavailable in regular
translation editing UIs, but which participants would like to see in a UI that
supports post-editing, 14 users said they would like to see improved glossaries
and dictionaries, with six wanting to be able to make glossary changes that would
be propagated throughout a document. Three suggestions related to MT and
involved improved display of provenance data (e.g. the origin of the suggested
translation), improved pre-processing, and dynamic changes to the MT system in
the case of a recurrent MT error that needs to be fixed many times during post-
editing. Other UI wishes included a global find-and-replace function, reliable
concordance features and grammar checking. Notably, these latter UI requests
are for features to support the general translation task, adjudged to be lacking in
users’ current tools (see Figure 6.3) despite two decades of TM tool development.
Figure 6.3 Tools used for translation and post-editing

Note: n = 246
Table 6.2 Keyboard shortcuts requested
Shortcut No.
Dictionary search 203

One-click rejection of MT suggestion 185
Web-based parallel text lookup 158
Change capitalization 149
Apply source punctuation to target 128
Add/delete spaces 124
Participants appear to be keen users of keyboard shortcuts: 29 per cent (70) of

241 participants use keyboard shortcuts often and 40 per cent (96) use them ‘very
often’, while only 5 per cent never use them at all. Of respondents, 80 per cent
(193) said that their productivity was improved by using keyboard shortcuts. For
specific operations required in MT post-editing and some operations not covered
by current default shortcuts, participants were asked whether a keyboard shortcut
might be useful. Responses are shown in Table 6.2. Proposed shortcuts are listed
in the left-hand column and the number of respondents who considered these
shortcuts useful in the right-hand column.
Dictionary search and the suggestion of a shortcut for web-based parallel text
look-up, while popular in survey responses, are not specific to post-editing. Post-
editing-specific responses included a 77 per cent preference (185) for a keyboard
shortcut that would allow a one-click rejection of an MT suggestion. This assumes
Table 6.3 Language-specific keyboard shortcuts requested
Shortcut No. of requests
Adjust word order 102

Change number (singular/plural) 99
Change gender 79
Change verb form 68
Add/delete preposition 67
Add/delete postposition 65
that the MT suggestion is automatically pasted to the edit window, of course,

which could be configurable in an editing interface. In XTM, for example, the
MT suggestion may either be automatically pasted or added electively using a
keyboard shortcut, depending on user settings. Incorrect letter casing is also often
problematic in MT output, reflected by the 62 per cent (149) who would like to
see a keyboard shortcut for changing capitalization.
Fewer participants consider our suggested language-specific keyboard shortcut
suggestions useful, possibly because of the large spread of target languages
among participants. The most popular suggested shortcut would change the
word order in a machine-translated segment (considered useful by 42 per cent,
or 102 participants), followed by a change in the grammatical number of a word
(e.g. from singular to plural). Further responses to suggested language-specific
shortcuts may be seen in Table 6.3. Notably, the suggestions for prepositions and
for post-positions do not apply to all languages, which may have led to lower
numbers considering these options useful. Additionally, participants may be
unable to measure usefulness without first testing these features in practice.
Participants expressed their opinions relating to these shortcuts in the open
comments sections of the survey. Of 125 commenters, 34 were in favour of
the shortcuts: ‘It seems obvious to me that all such keyboard shortcuts would
be useful. I reckon I use 15–25 keyboard shortcuts in each of the main CAT
and other productivity applications I use on a daily basis.’ Nine comments had
further suggestions for shortcuts, such as Internet-based text look-up, parallel text
searching and ‘copy source formatting to target’. Negative comments were made
by 41 participants, with many unable to understand how the shortcuts might work
in practice, while 18 participants had misgivings specific to one of their languages.
Several thought that manual changes would be easier or less time-consuming than
memorizing a large number of shortcuts – an opinion that recurs in the interviews.
Participants appeared to favour customizable shortcuts: 68 per cent (164) would
like to be able to adapt the UI functionality using macros or scripts; 52 per cent (125)
would like to be helped or guided in creating such a macro. Three comments expressed
a desire for instructions be clear and simple, and interoperability considerations to be
taken into account, so that macros from other programs (MS Word was suggested)
might work in the UI. Of 20 comments, all but one were positive about user-added
macros. Two commenters use AutoHotkey to set shortcuts globally on their systems,
but would like to be able to add program-specific shortcuts.
Presentation of TM matches and MT output
Of the 233 participants who completed this section of the survey, 81 per cent (189)
would like to be presented with confidence scores for each target text segment
from the MT engine. Of those 189, 70 per cent would like confidence scores to
be expressed as a percentage (like a fuzzy match score in a TM tool), while 25
per cent expressed a preference for colour-coding. If an MT suggestion were to
receive a higher confidence score than any fuzzy match available from the TM, 88
per cent of participants (205) said that they would nevertheless like to see both MT
suggestions and TM matches. Only three participants (just over 1 per cent) would
like to see the MT match only. Both of these findings suggest a lack of translator
confidence in MT output. This scepticism is also expressed by the 14 participants
who would like to see the TM match only, even when a higher-rated MT suggestion
is available, and in the choice by many participants of the lowest possible fuzzy
match value below which they would prefer to see MT output rather than a TM
fuzzy match. (The thresholds chosen by participants may be seen in Figure 6.4.)
But despite other evidence suggesting a low level of post-editor confidence in
MT, 80 per cent of respondents (186) would like to see ‘the best MT suggestion’
automatically appear in the UI target window when no TM match is available.
Of participants, 62 per cent (144) felt it would be useful if the editing
environment could combine the best sub-segments (or phrases) from the MT
system and the TM to produce a MT–TM (MTM) combined suggestion. Mixed
opinions were apparent among 47 participants commenting about this proposed
function; 21 commenters responded positively about a potential MTM match; and
9 commenters were not in favour of the feature, with one writing that it ‘seems
theoretically useful, but when really applied it (could) create confusion’.
Figure 6.4 TM fuzzy match thresholds below which an MT match is favoured

Note: n = 233
Five commenters had suggestions such as allowing the feature to be disabled,
while 87 per cent of respondents (203) said they would like to see the provenance of
MT or TM suggestions denoted by colour at a sub-segment level. The importance
of provenance and retention of metadata showing the origin of match suggestions
appeared clear across the whole survey.
Features to support post-editing

The section of the survey on potential functions to support post-editing was
completed by 231 participants completed. We have already outlined how some
participants expressed a desire to see dynamic, real-time improvements to MT
systems. Some work on this topic has been published by Alabau and colleagues
(2012: 20), who suggest that MT systems could use human post-edits as
‘additional information to achieve improved suggestions’. In our survey, 71 per
cent of respondents (164) said that their edits should be used to improve a client-
specific MT system, while 23 per cent (53) were unsure, with concerns expressed
in 42 comments. Four commenters were concerned about issues relating to client
confidentiality, while others resented further reuse of their translation work.
This intellectual property (IP) concern was expressed by one participant, who
wrote: ‘Who would pay a translator for his intellectual work in improving the
TM/MTM?’
Of participants, 69 per cent (159) would like to see edits not only used
to improve suggestions, but also to retrain an MT system in real time. Most
commenters felt positively about potential improvements, one writing that
‘[this] should be one of [the] main goals of MT, not only lower rates’. Again,
several participants would like to use this function electively. Three commenters
believe that the client should decide whether the content should be added to
the MT engine and five were not in favour of this function. One participant
felt that if it were possible to incorporate this function, it may lead to further
complications depending on the workflow and steps required for review or
approval: ‘If immediately incorporated, I’d like to know where each segment is
coming from (i.e. what is from [the] old MT engine, what is a recent addition/my
own work, what’s been reviewed as accurate, and what’s still pending.’ Another
commenter wondered how a system could learn only the ‘right’ changes (i.e.
changes to recurring incorrect phrases or terms), noting that ‘a too-generalized
auto-adaptation feature may create errors’.
Participants were asked how useful they would find two variants of IMT. Using
the first variant, the editing environment could dynamically alter pre-populated
MT suggestions depending on edits as the user moves through a segment, so that,
as the user edits the MT suggestion, the system would offer ‘context-dependent
completions’, adjusting the remainder of the target segment based on the user’s
edit (Langlais et al. 2002: 78). Of respondents, 48 per cent (111) considered that
this feature would be useful, with 28 per cent (65) unsure. Participants were more
certain that they would like to be able to turn this feature on and off, with 93 per cent
(215) requesting that the function could be used electively. Meanwhile, 46 per cent
(106) were in favour of a second variant of IMT, whereby the editing environment
could dynamically autocomplete segments translated from scratch based on MT
proposals – although 20 per cent (46) thought that this would not be at all useful. A
slightly higher proportion, 54 per cent (125), would like to see MT suggestions at a
sub-segment level, with 184 participants (80 per cent) saying that they would like to
see sub-segment MT suggestions provided as a dropdown list, with 35 per cent (64)
of those suggesting between two and three list items, and a further 34 per cent (63)
preferring the ability to customize the number of list items themselves.
The final questions related to user feedback on productivity – something
that is of great importance in the translation industry and is largely driving MT
deployment (DePalma et al. 2013) – and also revealed participants’ suspicion of
data dispossession. Some 70 per cent (162) would like to see their productivity
reported dynamically, such as in words per hour or percentage completed, as
long as this reporting function can be turned on or off and the information is for
their personal use only, while 48 per cent would like to see dynamic reporting of
their earnings. Among 58 comments about this proposed feature, 36 participants
consider this to be a great idea. One participant wrote:
Even after 6 years in the industry, I still find estimating time vs. fees to be
quite difficult. Even when I am able to view the source text beforehand to
make my estimate, I often misjudge the quantity or technicality of the work.
I think having an automated tracker would be fairer for me and the client.
Ten commenters specified that this sort of information should not be available to
the client (‘For client tracking – oh hell no’), and 13 would not be in favour of this
function at all, saying that it is unnecessary, will create more clutter and will put
post-editors under too much pressure.
Interview results
Forty-three survey participants agreed to waive their anonymity and make
themselves available for interview. We contacted 16 of them, choosing only
those who had post-editing experience and attempting to cover a wide range of
language pairs. Ten participants agreed to participate in follow-on interviews, all
but one of whom listed English as their source language. Interviewee profiles are
shown in Table 6.4.
Interface design
In response to the question ‘What existing software comes closest to your ideal
UI?’, four interviewees chose SDL Trados (all but one specified the Studio
version), four interviewees mentioned MemoQ, one chose SDLX and one chose
OmegaT. Informant D chose the SDLX tool, but only because she had been able
to customize the tool so as to link with online dictionaries. Interviewees were
asked: ‘What do you think is most important: simplicity in the UI or a feature-rich
Table 6.4 Interviewee profiles
Post-editor Languages Post-editing Tools

experience
A English > French 2 years SDLX

B English > Portuguese 6 years SDLT; GTT
C English > Spanish 2 years MemoQ; SDLX; SDL
Trados; Worldserver;
others
D English > French 5 years SDLX; Trados (2007 and
Studio); TagEditor;
Idiom
E English > Italian 2 years Wordfast Pro or SDL
Trados Studio 2011
F English > Spanish 9 months MemoQ; Trados (2007
and Studio); TagEditor;
Idiom
G English, German, Spanish, Various projects OmegaT
Catalan > Italian over years
H Russian > English 5 months or so SDL Trados
I English > Finnish 1 year Trados; Wordfast; Idiom
Worldserver
J English > Arabic 2 years SDL Trados; MemoQ;
WordFast; Catalyst
(SDL Trados 2009 most
efficient)
UI?’ Three interviewees chose ‘feature-rich’ and three, ‘simplicity’, but for many
this was not an appropriate distinction; rather than having many or few items
onscreen, they considered it important to have ‘the right’ features. Informant B
complained of too much information displayed to the user in the SDL Trados
interface, saying ‘you have to use a big screen to be able to leverage from all of
that information’.
Several informants felt that the solution to on-screen clutter is in having a highly
customizable UI. For H, ‘what I’d like would be more opportunity to build buttons
into the interface myself’. He continued: ‘There are some functions that I want
to use repeatedly, and I have to go through various dropdown menus (to access
them).’ Another possible approach could be to create a UI that adapts to the user or
one that presupposes a learning curve whereby more functionality may be revealed
as users gain experience with the new interface. Tidwell (2006: 45) recommends
designing a UI, particularly for new users, that hides all but the ‘most commonly
used, most important items’ by default. Informant C explained that users’ needs
may change as they become familiar with the UI:
I have a lot of difficulties learning shortcuts at the beginning, but then after 6
months using the same tool, you find that those buttons, you don’t need any
more, so maybe something that could be customized to the user.
F added that, for him, performance ranks higher than the look of a UI and
additional functionality is useful only if allied with performance. ‘In the end,’
he said, ‘what decides is how fast we can work with it.’ In his office, although F
considers that SDL Trados Studio has good features, ‘we use MemoQ because
it’s quicker’. Good performance makes this compromise worthwhile: ‘We miss
some features, but we make up for it in speed.’ Referring to ease of use of editing
interfaces, F said that many of the edits he makes ‘are very easy to see but very
cumbersome to make with the common tools’:
When a segment is wrong, you have to start all over again from scratch, then
the user interface has no influence whatsoever, but if a segment is almost
right, these little things that could make it better, those are the things that
could really speed up the process.
Current post-editing problems

Interviewees were asked about common edits they make during post-editing
and possible ways in which to support these changes in the UI. Six informants
mentioned word-order changes. Four informants reported problems with
terminology, such as term updates since the MT engine was trained. Three
informants mentioned grammar problems generally, with D asserting that, for her,
the ‘main pitfall of Statistical Machine Translation (SMT)’ is ‘all the agreement:
all the gender, number, conjugation’. Other orthographical and grammatical pain
points mentioned were letter case, prepositions, number and gender. F said that
these kinds of changes ‘are the most frustrating for us because it’s mechanics, and
if it’s mechanic, there must be a way it could be done by a machine’. He noted that
gender changes should also include ‘other elements . . . like associated articles
and the words surrounding the noun’.7
Three interviewees would like to see automated language-specific changes to
word order, with B commenting that ‘one of my dream features that I haven’t
seen so far is changing word order’. She said that, in Portuguese, ‘we use an
adjective after the noun and it’s quite the opposite in English, so that is a really
common error in machines that are not well-trained’. Two interviewees requested
that highlighted words could be swapped around within a segment. E said that she
would like to drag and drop words into the correct order, while G thought such
a feature would be superfluous: ‘It will always be quicker doing that with a few
keystrokes rather than clicking some button or special shortcut.’
Two interviewees requested propagation of edits in subsequent segments: one
would like to see a feature such as autosuggest in SDL Trados Studio (a feature
whereby a word or phrase from the TM or glossary is suggested to the user based
on context while the user types); another suggested a list of alternative translations
for a word when the cursor hovers over it. When interviewees were asked more
generally about pain points in the post-editing task, five returned to the topic of
MT quality. D stated: ‘I will only work with clients who customize their MT. I
will not work with anybody who just sends something to Google Translate and
says “go ahead and post-edit this”.’ A finds that, because, in her current UI, MT
is highlighted in violet, she tends to ‘jump automatically to the target segment as
if I were reviewing’. Even though she finds it ‘really is important to look at the
source first’, the highlighting makes it ‘difficult to focus on this ideal process of
looking at the source segment before looking at the target’.
Four participants were dissatisfied with the lack of confidence scores for MT.
Interviewee I complained that, when the MT is completely wrong, ‘it’s more work to
post-edit machine translation than just to translate from scratch’. Nine interviewees
would like to see improved terminology features. B requested ‘a shortcut to get
different (language-specific) variations for the same term’. She added that ‘having a
glossary that automatically produces variation regarding gender and number; I think
that would be a killer glossary’. Four would like to make global term changes, with D
suggesting a scenario in which she could update the second part of a project, ‘taking
into account what you’ve added during the first part – that would be amazing’.
Features to support post-editing

When asked in interviews about combined sub-segment matches from MT and
TM, all but one of the interviewees responded positively. Some were concerned
with how it would work in practice. An advantage of the interview stage was
the possibility of yielding these more detailed and considered opinions from the
participants. B felt it would be useful ‘where you don’t have a dedicated glossary
but you have a really rich TM’. J, however, believes that this is a ‘far-fetched
feature that will complicate things further’.
All interviewees would like to see edits used to improve an MT system in real
time, although five would like these improvements communicated to the user
(mostly using colour-coding) and five would like it to happen without notification.
F said: ‘It’s very important to know where a segment has been assembled from,
because you have to take a different approach to post-edit it.’ E suggests feedback
if the user hovers the cursor over a word that has been updated and C would like
to see prompting, because unsupervised improvements could be ‘very dangerous’.
B and J felt that notifications might delay them. J said: ‘I don’t want to go through
too many details while I work . . . I just want the suggestion in order to be used as
quickly as possible.’
Interviewees were asked about a function whereby the MT system would
provide suggestions that would be dynamically altered as they type, based on
their edits. Six interviewees were in favour of such a function, whereas four
interviewees thought that this would be distracting or might delay their translation.
D said: ‘Although I type fast, I still look quite a lot at my keyboard, so I wouldn’t
see the edits as they happen while I’m typing.’
Discussion
The survey and interviews elicited user input to potential future UIs for post-
editing, and identified pain points in the post-editing process. They also revealed
a prevailing level of frustration among users concerning fundamental support for
the translation task quite aside from post-editing. This finding is in keeping with
recent workplace studies such as that carried out by Ehrensberger-Dow (2014) in
which 19 hours of screen recordings were collected in the translation workplace
for analysis. In their study, Ehrensberger-Dow and Massey (2014) found that
certain features of the text-editing software used by the translators were slowing
down the process, such as that the small window in which the translators had
to type resulted in them frequently losing their position when they had to shift
between areas of the UI to obtain other information.
In much of the present survey, and in particular in the interviews, we focused
on post-editing-specific problems and requirements. We found evidence of
scepticism towards MT based on perceptions of MT quality and a feeling of
dispossession related to the translator’s work being used for MT retraining.
Survey respondents stated a preference for even low-threshold TM matches (less
than 65 per cent) over MT output, despite previous findings that such TM matches
require more effort to edit than MT output would (O’Brien 2006; Guerberof 2008).
Scepticism about MT might also be behind survey respondents’ enthusiasm for
one-click rejection of MT suggestions. Both survey respondents and interviewees
expressed frustration at having to make the same post-edits over and over again,
and would like to see ‘on the fly’ improvements of MT output based on their
edits – a scenario that is increasingly plausible owing to recent breakthroughs in
SMT retraining speeds (Du et al. 2015). Nonetheless, MT and UI developers need
to find an efficient method of making incremental improvements to MT output
in real time, to lessen the frustration of post-editors who are currently obliged to
make the same edits repeatedly.
A further issue that emerged in discussions of the integration of MT with TM
related to post-editors’ strong desire to know the provenance of data that would
be reused in such scenarios. Retaining this level of provenance of data would,
however, require both user tracking and careful retention of metadata, from both
TMs and training data used in SMT, the latter of which would be particularly
technically demanding.
Conclusion
In this chapter, we have reported on the results of an online survey with 231 full
participants and a series of interviews with 10 informants, focusing on the task of
post-editing MT and associated UI requirements. Our results provide an update
to survey work by Lagoudaki (2006, 2008) on TM tools, and what users consider
important and desirable in a translation editor. The focus of the study was on post-
editing and MT, but an unexpected pain point was continued dissatisfaction with
translation tools in general. Despite updates of popular tools in recent years and
new UI features appearing in tools such as Matecat, Casmacat and Lilt,8 our study
identified a perceived lack of development within existing translation interfaces,
with survey and interview participants complaining of long-standing issues with
their current interfaces. This highlights a lack of human–computer interaction
(HCI) input in translation tool development and design, and would suggest a real
need for input from HCI experts.
Our modus operandi could be criticized on the basis that participants were
asked to comment on features that are not yet available in commercial tools.
Sleeswijk Visser and colleagues (2005: 122) warn that such user responses may
‘offer a view on people’s current and past experiences’, rather than expose ‘latent
needs’. While we accept that such responses have their drawbacks, we consider
user input very much worthwhile as part of a predesign survey that may lead to a
new step in interactive translation support. We hope that some latent needs may
also have been revealed during the interview stage and consider that unforeseen
requirements may arise during user testing of any prototype UI. We further hope
that this research may contribute to translation editor interfaces optimized for
post-editing, which reflect the functional and design needs of users.
Acknowledgements
This research is supported by the Science Foundation Ireland (Grant 12/CE/I2267)
as part of the Centre for Next Generation Localisation (CNGL, http://www.cngl.ie)
at Dublin City University. The authors would like to thank the companies that helped
to promote the post-editing survey: Alchemy/TDC, Lingo24, Pactera, Roundtable,
VistaTEC and WeLocalize. We are grateful to Brian Kelly of TimeTrade Systems
for advice on specifications creation, and to Professor Dave Lewis and Joris Vreeke
for feedback on the ensuing specifications. Finally, we would like to thank Dr
Johann Roturier (Symantec), Morgan O’Brien (Intel) and Linda Mitchell (Dublin
City University) for comments and advice on survey questions.
Notes
1 Interim results only from the survey were published in Moorkens and O’Brien (2013).
2 A confidence measure is an automated estimate of raw MT quality and may be presented
at a segment or sub-segment level (Blatz et al. 2004). At the segment level, they may
inform a user of the likely usefulness of the raw MT; at the sub-segment level, they may
inform a user about incorrect, or possibly incorrect, translations (Alabau et al. 2013).
There have been various suggestions as to how to estimate and display confidence scores
(Blatz et al. 2004; González-Rubio et al. 2010), but a confidence estimation feature has
not yet been included in a commercial translation editor.
3 The number of participants who completed a section will be presented in reporting that
section. Where percentages are given, these represent the percentage of the participants
who completed that particular section. (In such cases, absolute values are given in
parentheses.)
4 In the IT localization sector, English is the main source language (Kelly et al. 2012).
5 Lagoudaki (2006) found that SDL Trados was also the most widely used tool within her
2006 cohort, with 51 per cent reporting that they used the tool regularly.
6 On the contrary, Lagoudaki (2006) reported that company employees were more likely
than freelancers to use multiple tools.
7 Many of these suggested features are similar to those proposed in the survey, which begs
the question: did the survey prime participants for these interviews or set them thinking
on the topic of a post-editing UI? It is unlikely that they remembered an online survey
in detail from one month previously (and once closed, the survey was not available to
browse), but it is quite possible that specific suggested features remained in mind at the
time of the interviews.
8 See http://www.lilt.com [accessed 12 March 2016].
References
Alabau, V., R. Bonk, C. Buck, M. Carl, F. Casacuberta, M. García-Martínez, J.
González, P. Koehn, L. Leiva, B. Mesa-Lao, D. Ortiz, H. Saint-Amand, G. Sanchis
and C. Tsoukala. 2013. ‘Advanced Computer-Aided Translation with a Web-Based
Workbench’. In Proceedings of Machine Translation Summit XIV Workshop on Post-
Editing Technology and Practice. Stroudsburg, PA: Association for Computational
Linguistics, 55–62.
Alabau, V., L. A. Leiva, D. Ortiz-Martínez and F. Casacuberta. 2012. ‘User Evaluation
of Interactive Machine Translation Systems’. In Proceedings of the 16th Annual
Conference of the European Association for Machine Translation (EAMT). Stroudsburg,
PA: Association for Computational Linguistics, 20–23.
Aziz, W., S. Castilho de Sousa and L. Specia. 2012. ‘PET: A Tool for Post-Editing and
Assessing Machine Translation’. In Proceedings of the Eighth International Conference
on Language Resources and Evaluation (LREC’12). Paris: ELRA, 3982–7.
Blatz, J., E. Fitzgerald, G. Foster, S. Gandrabur, C. Goutte, A. Kulesza, A. Sanchis and N.
Ueffing. 2004. ‘Confidence Estimation for Machine Translation’. In Proceedings of
the Twentieth International Conference on Computational Linguistics (COLING’04).
Cettolo, M., N. Bertoldi and M. Federico. 2013. ‘Cache-Based Online Adaptation for
Machine Translation Enhanced Computer Assisted Translation’. In Proceedings of
Machine Translation Summit XIV. Stroudsburg, PA: Association for Computational
Linguistics, 35-42.
Dara, A. A., S. Dandapat, D. Groves and J. van Genabith. 2013. ‘TMTprime: A
Recommender System for MT and TM Integration’. In Proceedings of the 2013
NAACL HLT Demonstration Session. Stroudsburg, PA: Association for Computational
Linguistics, 10–13.
de Almeida, G. 2013. Translating the Post-Editor: An Investigation of Post-Editing Changes
and Correlations with Professional Experience across Two Romance Languages. PhD
thesis. Dublin City University. Available at: http://doras.dcu.ie/17732/1/THESIS_G_
de_Almeida.pdf [accessed 12 March 2016].
de Souza, J. G. C., M. Turchi and M. Negri. 2014. ‘Machine Translation Quality
Estimation across Domains’. In Proceedings of the 25th International Conference
on Computational Linguistics (COLING 2014): Technical Papers. Stroudsburg, PA:
Association for Computational Linguistics, 409–20.
DePalma, D. A., V. Hegde, H. Pielmeier and R. G. Stewart. 2013. The Language Services
Market: 2013. Boston, MA: Common Sense Advisory.
Doherty, S., and S. O’Brien. 2013. ‘Assessing the Usability of Raw Machine Translated
Output: A User-Centred Study Using Eye Tracking’. International Journal of Human-
Computer Interaction 30(1): 40–51.
Du, J., J. Moorkens, A. Srivastava, M. Lauer, A. Way and D. Lewis. 2015. D4.3:
Translation Project-Level Evaluation. FALCON Project EU FP7. Available at: http://
falcon-project.eu/wp-content/uploads/2015/11/FALCON-D4.3_Submit.pdf [accessed
12 March 2016].
Ehrensberger-Dow, M. 2014. ‘Challenges of Translation Process Research at the
Workplace’. MonTI Monographs in Translation and Interpreting 7: 355–83.
Ehrensberger-Dow, M., and G. Massey. 2014. ‘Cognitive Ergonomic Issues in Professional
Translation’. In J. W. Schwieter and A. Ferreira (eds). The Development of Translation
Competence: Theories and Methodologies from Psycholinguistics and Cognitive
Science. Newcastle: Cambridge Scholars, 58–86.
González-Rubio, J., D. Ortiz-Martínez and F. Casacuberta. 2010. ‘On the Use of Conﬁdence
Measures within an Interactive-Predictive Machine Translation System’. In Proceedings
of the 14th Annual Conference of the European Association for Machine Translation
(EAMT). Stroudsburg, PA: Association for Computational Linguistics. Available at: http://
www.mt-archive.info/EAMT-2010-Gonzalez-Rubio.pdf [accessed 12 March 2016].
Guerberof, A. 2008. Productivity and Quality in the Post-Editing of Outputs from
Translation Memories and Machine Translation. Minor dissertation. Universitat Rovira
i Virgili, Tarragona.
Guerberof, A. 2013. ‘What Do Professional Translators Think about Post-Editing?’.
Journal of Specialised Translation 19: 75–95.
Kelly, N. 2014. ‘Why So Many Translators Hate Translation Technology’. The Huffington
Post. 19 June. Available at: http://www.huffingtonpost.com/nataly-kelly/why-so-
many-translators-h_b_5506533.html [accessed 12 March 2016].
Kelly, N., D. DePalma and V. Hegde. 2012. Voices from the Freelance Translator
Community. Boston, MA: Common Sense Advisory.
Koehn, P. 2009. ‘A Process Study of Computer-Aided Translation’. Machine Translation
23(4): 241–63.
Koehn, P., V. Alabau, M. Carl, F. Casacuberta, M. García-Martínez, J. González-Rubio, F.
Keller, D. Ortiz-Martínez, G. Sanchis-Trilles and U. Germann. 2015. CASMACAT Final
Public Report. CASMACAT EU FP7. Available at: http://cordis.europa.eu/docs/projects/
cnect/6/287576/080/reports/001-finalpublicreport.pdf [accessed 12 March 2016].
Koponen, M. 2012. ‘Comparing Human Perceptions of Post-Editing Effort with Post-
Editing Operations’. In Proceedings of the Seventh Workshop on Statistical Machine
Translation. Stroudsburg, PA: Association for Computational Linguistics, 181–90.
Lagoudaki, E. 2006. ‘Translation Memories Survey 2006: Users’ Perceptions around TM
Use’. Paper presented at Translating and the Computer 28. London, 16–17 November.
Lagoudaki, E. 2008. Expanding the Possibilities of Translation Memory Systems: From the
Translator’s Wishlist to the Developer’s Design. PhD thesis. Imperial College London.
Langlais, P., G. Lapalme and M. Loranger. 2002. ‘Transtype: Development–Evaluation
Cycles to Boost Translator’s Productivity’. Machine Translation 15(2): 77–98.
McBride, C. 2009. Translation Memory Systems: An Analysis of Translators’ Attitudes
and Opinions. MSc thesis. University of Ottawa. Available at: https://www.ruor.
uottawa.ca/handle/10393/28404 [accessed 12 March 2016].
Moorkens, J., and S. O’Brien. 2013. ‘User Attitudes to the Post-Editing Interface’. In
Proceedings of Machine Translation Summit XIV Workshop on Post-Editing Technology
and Practice. Stroudsburg, PA: Association for Computational Linguistics, 19–25.
Moorkens, J., and S. O’Brien. 2014. ‘Post-Editing Evaluations: Trade-offs between Novice
and Professional Participants’. In Proceedings of the 18th Annual Conference of the
European Association for Machine Translation (EAMT 2015). Allschwil: EAMT, 75–81.
Moran, J., and D. Lewis. 2011. ‘Unobtrusive Methods for Low-Cost Manual Evaluation
of Machine Translation’. In Proceedings of Tralogy 2011. Paris: Centre National de
la Recherche Scientifique. Available at: http://lodel.irevues.inist.fr/tralogy/index.
php?id=141 [accessed 12 March 2016].
O’Brien, S. 2006. ‘Pauses as Indicators of Cognitive Effort in Post-Editing Machine’.
Across Languages and Cultures 7(1): 1–21.
Ozcelik, D., J. Quevedo-Fernandez, J. Thalen and J. Terken. 2011. ‘Engaging Users in the
Early Phases of the Design Process: Attitudes, Concerns and Challenges from Industrial
Practice’. In Proceedings of the 2011 Conference on Designing Pleasurable Products
and Interfaces (DPPI ‘11). New York: ACM, 1–18.
Sleeswijk Visser, F., P. J. Stappers, R. van der Lugt and E. B. N. Sanders. 2005.
‘Contextmapping: Experiences from Practice’. CoDesign 1(2): 119–49.
Somers, H. 1998. ‘Machine Translation: Applications’. In M. Baker (ed.). Routledge
Encyclopaedia of Translation Studies. London: Routledge, 136–9.
Tidwell, J. 2006. Designing Interfaces. Sebastopol, CA: O’Reilly.
7 Issues in human and automatic
translation quality assessment
Stephen Doherty
Introduction
The main goal of translation quality assessment (TQA) is to ensure a specified level
of quality is reached, maintained and delivered to the client, buyer, user, reader,
etc., of translated texts. Translation quality assessment is also crucial to formative
assessment for translators in training, for professional certification and accreditation,
and for recruitment and screening of translators. In research settings, TQA can be
used to measure performance of translation technologies – especially machine
translation (MT) – to assess translator productivity in technologized workflows and
to study other related constructs, such as post-editing effort and quality estimation,
etc. This list of usage scenarios for TQA is by no means exhaustive, but serves to
highlight the diversity of its applications, from fundamental and applied research
projects, to everyday industry pricing and public consumption.
Given its importance, it is not surprising that TQA has been a topic of much
debate in translation studies in general and translation technology in particular,
as well as throughout the translation and localization industry. There are
commonalities in the theoretical discussion of translation quality – most notably,
in the dichotomy between source-oriented notions of ‘accuracy’ and ‘adequacy’,
on the one hand, and target-oriented notions of ‘fluency’ and ‘acceptability’, on
the other, each in their many terminological guises. In practice, however, resource
constraints and the adoption of a necessarily pragmatic approach in many areas
mean that TQA processes vary considerably and have many limitations. At the
same time, the evolution and widespread adoption of translation technologies –
especially MT – have resulted in a plethora of typically implicit and differently
operationalized definitions of quality and respective measures thereof.
This chapter does not attempt to reconcile the diversity and occasional
incompatibility of these approaches and applications; rather, it aims to highlight
a number of universal issues that arise in TQA, especially in contemporary
technologized environments. (For a review of the wider impact of translation
technologies, see Doherty 2016.) Such issues have to do with the decisions that
humans make (or avoid making) in designing translation evaluations and with
the methodological rigour of the TQA they subsequently carry out. They concern
the (human-produced) materials used in TQA and the people chosen to act as
evaluators. And they involve perceived tensions between human subjectivity and
132 Stephen Doherty
machine objectivity. In short, they are profoundly human issues that arise in TQA,
even when such TQA is conducted by machines on machine output.
Of course, many of these issues are not unique to machine environments; they
also arise in more ‘traditional’ TQA, as addressed in translation studies for decades,
and they arise in contemporary industry settings, where levels of automation vary
considerably. This chapter thus starts with a brief critique of approaches to translation
quality and its assessment in the translation studies literature. It then surveys the
approach to TQA taken in contemporary MT research, before viewing TQA from
the perspective of the translation industry. At each turn, it identifies shortcomings in
TQA approaches, for example in the lack of explicit operationalization of concepts
and in non-adherence to established standards upheld in test theory – namely, those
related to validity, reliability and the selection of evaluators. The second part of
the chapter homes in on these standards, identifying ways in which psychometric
principles can be (and often are) flouted in TQA, and pointing towards ways in
ich training and best practice could be better supported.
Translation quality assessment in translation studies

Translation studies has tended to approach translation quality from a theoretical
and case-study perspective, rather than a large-scale experimental one – a trend also
evident in comparable disciplines such as psycholinguistics and second-language
acquisition. The translation studies approach has yielded much debate, some of
which is centred on the relative merits of source vs target orientation, and much of
which acknowledges the importance of the purpose and function of a translation
in assessing its quality (see House 2009 for an overview). While a universal, and
entirely objective, approach to TQA would be difficult to argue for – in part, because
of the complexity of the translation process, but also because of the diversity of texts,
languages, directions and applications used to inform the TQA debate (see Nord
1991; House 1997; Schäffner 1998; Reiss 2000; Williams 2001) – this does not mean
that translation studies scholars should not aim for rigour in their treatment of the
subject. The challenges of TQA in TS have been signalled by a number of scholars,
however, who cite lack of research (e.g. Cao 1996a, 1996b; Hatim and Mason 1997),
error proneness and subjectivity (e.g. Bassnett-McGuire 1991; Snell-Hornby 1992;
Hönig 1998; Bowker 2000; Koponen 2012), an absence of systematic approaches
(e.g. Sager 1989; Bassnett-McGuire 1991; Hönig 1998), and inconsistency and
confusion in terminology (Brunette 2000) as some of the many issues requiring
resolution if we are to move forward. It is not even self-evident who should evaluate
translations, with Reiss (2000: xi) remarking that ‘the standards most often observed
by critics are generally arbitrary, so that their pronouncements do not reflect a solid
appreciation of the translation process’.
While some scholars have focused on the analysis of linguistic features to
approach TQA in a more objective way (e.g. Hatim and Mason 1990, 1997;
Baker 2011), there is a growing awareness that more empirical work is needed
in this area (e.g. Waddington 2004; Anckaert et al. 2008). Reiss (2000) too
believes that more objective and evidenced-based TQA would inform evaluators
Human and automatic translation quality 133
and be a useful pedagogical tool for translators themselves. With the more recent
availability of large volumes of linguistic corpora, many scholars have also used
these rich resources to guide TQA processes and to educate translation students
and evaluators (e.g. Bowker 2001, 2003; Uzar 2004).
It is argued here that the volume of new translation studies research in this area
should not be of primary concern; rather, we should be concerned with its methodical
rigour and generalizability, so that it can address the need for validated and reliable
empirical findings, and thus bridge the gap between translation studies and the
industry. While it may be clear for some that ‘we need more objective data both on
linguistic and stylistic norms of the given languages in order to develop more objective
assessment criteria’ (Baer and Bystrova-McIntyre 2009: 160), the methods of defining
and measuring objective elements are not an inherent skillset within the translation
studies discipline or for practitioners in the industry; rather, they must be borrowed
from other disciplines that have already incorporated standards from experimental
approaches, for example psycholinguistics. Thus Anckaert and colleagues (2008)
compare translation studies with the domain of language assessment, which has a
more developed incorporation of psychometrics and empirical studies that verify the
reliability and validity of language proficiency tests themselves. Relatedly, Eyckmans
and colleagues (2009: 73) find similar issues in educational and professional
organizations in relation to translator certification and accreditation testing, where ‘the
reliability and validity of those tests remain underexplored’.
Translation quality assessment in machine translation research

As House (2009: 222) points out, TQA in translation studies is very bound up
with theories of translation itself. In contrast, while the field of MT has adopted
TQA as a central aspect in its development, theoretical discussions of translation
and translation quality are largely unknown in this field. Research in MT, like
industry practice, instead frames TQA as part of a larger process – one in which it
is a means to an end and not an end in itself. The pragmatism that this entails, in
contrast to the more principled approach sought after in translation studies, is not
without its own shortcomings, however.
There are two main approaches to MT evaluation: the first relies on humans
to evaluate MT output; the second attempts to automate this evaluation. Human
evaluation of MT is most commonly carried out under the adequacy and fluency
paradigm, in which adequacy (also known as ‘accuracy’) is understood as the
extent to which the MT output captures the meaning of the source-language input,
and fluency (also known as ‘intelligibility’) is the extent to which the MT output
is a ‘good’ exemplar of the target language. In such scenarios, human evaluators
typically rate MT output segment by segment (i.e. without context) and at sentence
level, with the source text segment presented alongside. Frequently, an ordinal scale
is used, whereby human evaluators can indicate, for example, that the MT output
captures ‘all of the meaning’, ‘most of the meaning’ . . . ‘none of the meaning’,
etc., of the source segment (Koehn 2010: 218–20). Monolingual evaluations are
also possible, in which the evaluator does not see the source-language input, but
134 Stephen Doherty
may see a human-produced reference translation (the equivalent of a ‘fair copy’
at sentence level) that is supposed to capture the meaning of the source-language
segment. Human evaluation of MT is resource-intensive and it can take much
longer than human evaluation of human translations, because MT output quality
can vary to a greater extent.
Within the MT community, there is growing awareness of other limitations of
human evaluations as currently conducted in MT research. These include issues
related to forced-choice dichotomies of good vs bad translation (see, e.g., Blatz
et al. 2003; Doherty et al. 2010) and the use of interval scales (e.g. Koehn and
Monz 2006; Specia and Farzindar 2010; Bojar et al. 2013; Graham et al. 2013).
Moreover, the subjectivity of human evaluators has also been highlighted (see, e.g.,
Koponen 2012; Turchi et al. 2014). It is, however, the need to perform frequent,
low-resource-iterative evaluations on the development of MT systems that has,
more than any other factor, led to the popularity of the automatic alternative to
human evaluation of MT.
Automatic evaluation mostly relies on automatic evaluation metrics (AEMs),
which are scripts or software programs that perform TQA using an explicit and
formalized set of linguistic criteria to evaluate the MT output, typically against a
human reference translation or ‘gold standard’. Popular AEMs include the bilingual
evaluation understudy (BLEU) (Papineni et al. 2002), general text matcher (GTM)
(Turian et al. 2003) and translation edit rate (TER) (Snover et al. 2006). Implicit
in AEMs is the assumption that translation quality can be evaluated on the basis
of a narrow range of prescribed linguistic features matching at word and sentence
level. This approach cannot subscribe to concepts of accuracy and fluency, because
AEMs are not (yet) sophisticated enough to make such judgements; instead, many
AEMs make use of the information-retrieval concepts of precision and recall,
building them into metrics of varying complexity, and operating usually on single
words (unigrams) and, to a lesser extent, longer n-grams.
In this context, precision is usually understood as the fraction of words in the MT
output that are correct (as indicated by the gold-standard reference translation), and
recall is the fraction of correct words (also indicated by the gold-standard reference
translation) that are produced in the MT output. If a reference translation were to
contain ten words and a given MT system to produce an eight-word translation
(for the same source-language segment), all of which appeared in the reference,
we would say that the precision is ‘8/8’ (i.e. 100 per cent) but recall is only ‘8/10’
(80 per cent), because while all the words produced by the system were ‘correct’,
it failed to produce two other ‘correct’ words. Other AEMs (e.g. TER) measure
the minimum number of edits (called ‘edit distance’ or ‘edit rate’) necessary to
make the MT output match the human reference translation. By making these
comparisons, AEMs assign numeric scores that can be used for iterative system
velopment, comparative performance and, more recently, industry applications.
Correlations between AEMs and human TQA scores vary considerably, and
should be interpreted with care: the most popular AEM, BLEU, for example, has
been shown not to correlate with other measures of translation quality (Callison-
Burch et al. 2006) and is commonly misused at sentence level even though it is
intended for document-level evaluation (Papineni et al. 2002). Despite known
difficulties with AEMs – not least the fact that ‘it is often the case that the results
of manual and automatic evaluations do not agree’ (Labaka et al. 2014: 97) –
AEMs remain the metrics of choice in MT research where human evaluation is not
widely applied (Labaka et al. 2014). As well as being used in MT research, AEM
scores are beginning to have more impact in industry applications, where they are
now used to indicate the quality of potential output from client engines built using
cloud-based MT services – yet a lack of awareness of what they can and cannot
measure remains, and the consequent misuse of these metrics can easily occur.1
Another risk associated with AEMs is that users could somehow find them
more ‘objective’ than other forms of (human) evaluation, on the basis that (albeit
subjectively programmed) AEMs make comparisons between machine outputs
and reference translations using explicit procedures that do not vary depending
on who is using them. Subjectivity is never eliminated, however, because the
human reference translation is itself a trace of the subjective judgement of a single
human translator. This problem is mitigated somewhat by versions of AEMs that
allow for multiple human references to be used at the one time, showing a general
acceptance in MT circles that there can be more than one ‘correct’ translation – a
belief long since accepted in the translation studies community – but even then we
are still dealing with multiple subjective human judgements. In addition, even the
choice of one AEM over another (or over other TQA alternatives) is still somewhat
subjective. That said, in terms of consistency, one can achieve the same results each
time using the same AEMs on the same data once the parameters are fixed. This,
however, leads to a further issue, because there are now many different features
and variations of the metrics – especially in-house modifications, extensions and
adjusted weightings – which are not systematically made explicit in publications,
and which can readily lead to different results being generated for the same data
and AEMs. Lastly, frustrations can arise when AEM scores do not equate to
clients’ or buyers’ expectations of quality even though certain metrics have shown
various levels of correlation with human judgement, for example GTM and human
post-editing (see, e.g., Tatsumi 2009; Tatsumi and Roturier 2010; Koponen 2012).
Similar questions arise regarding the data used in MT applications, especially
in the data-driven approaches in the popular statistical and hybrid systems that
dominate today’s research and industry landscapes. Here, too, the absence of an
explicit epistemological stance can be explained by the practicality and problem-
based approaches in publications and industry applications. Empirical research
and development in MT relies on human-annotated data or the (semi-)automated
collation of human-created or -annotated data in one form or another. For MT,
this typically comes in the form of mono-, bi- and multilingual corpora – often
from translation memory (TM) data – and/or linguistic rules created manually
by linguists or learned by a system using machine-learning methods. Of note in
the literature is the lack of awareness and interest in the source of these data, on
which MT and most natural-language processing (NLP) applications are entirely
dependent. Sources such as Kenny (2011) and Brunello (2012), for example,
question the ‘shared’ nature and ownership of linguistic resources when human
136 Stephen Doherty
translations are used to train MT systems, and, by extension, raise concerns about
the quality of these resources (at which we look shortly).
While efforts have been made to address concerns about poor and uncertain
quality in the corpora used for MT, considerable issues remain and are
propagated by the reuse of ‘big data’, the contents of which have not been fully
validated, for instance, using TQA. A case in point is large-scale MT shared
tasks such as the Workshop in Machine Translation (WMT), a commendable
collaborative venture in which MT researchers test and develop their systems
against others from all over the world (see, e.g., Bojar et al. 2013, 2014). These
campaigns typically involve the provision of existing corpora (e.g. Europarl,
News Commentary, Common Crawl, Gigaword, Wiki Headlines and the UN
Corpus), as well as WMT-commissioned translations with which MT systems
can be trained and tested, etc. Among others, Lommel and colleagues (2014)
highlight considerable errors and misgivings in WMT data. They note ‘loose
translations that preserved the broad sense, but not the precise details’ (Lommel
et al. 2014: 168) and ‘the poor level of translation evident in the WMT human
translations’, in which ‘there were numerous instances of basic mistakes and
interpretive translation that could generally be considered poor references’,
and difficulties arising from the source text meaning being unclear and out of
context (Lommel et al. 2014: 171).
Once WMT participants submit their systems’ outputs to the shared task, a
large-scale evaluation takes place during a set window, wherein individual
untrained participants (the researchers themselves) are assigned anonymized MT
output to be evaluated. These evaluations are usually ranked (see, e.g., Callison-
Burch et al. 2012), which means that the evaluator is presented with several MT
outputs and the source text segment, and must rank them from best to worst on a
scale of 1–5. It is often the case that the best translation is still not of high quality,
but is better than the others in terms of the given criteria, such as adequacy and/or
fluency, which are often left undefined.
The overall design of the evaluation also has a significant impact. Large-scale
evaluation campaigns are carried out via makeshift or static TQA platforms, such
as Appraise (Federmann 2010), which allow customization in timing, presentation
and collection of user activity data. Crowdsourcing approaches are also employed,
in which self-selected participants perform the evaluation for a fee, for example
using Amazon’s Mechanical Turk (Snow et al. 2008; Callison-Burch 2009). We
return to design issues in evaluation projects and the selection of evaluators later
in the chapter.
Translation quality assessment in industry

Several recent surveys of the industry provide insight into common practice in
relation to TQA (e.g. Doherty et al. 2013; Gaspari et al. 2014). From a large-
scale survey, Doherty and colleagues (2013) report that TQA is mostly used for
evaluating translators (e.g. new hires, professional development, working with
freelancers and outsourcing), acceptance checking, training and pricing. In terms of
TQA tools, most respondents state that they use those built into existing computer-
assisted translation (CAT) tools; in terms of frequency, TQA is carried out by the
majority, with regular testing and random checks less common. Interestingly, 70
per cent of respondents expressed a need for improvements to be made to current
TQA approaches and methods. Similar results can also be found in a comparable
survey described in Gaspari and colleagues (2014), which focuses on freelance
translators, academics and language service providers (LSPs).
Translation quality assessment in the translation industry is typically dominated
by human evaluation and semi-automated methods using deductive error-based
typologies that are employed to test randomly selected samples of translations. In
its simplest form, spreadsheets can be used, often with templates or ‘scorecards’,
while more advanced and tool-based TQA methods have become more widespread,
especially in the localization and IT industries. Error analysis follows a deductive
approach by counting linguistic and extralinguistic (e.g. formatting) errors and
assigning points based on their severity (e.g. minor, major, critical). Depending
on the user and usage scenario (e.g. client, external requirements, job size), a
quality threshold is set, at 95 per cent for example, above which the translation will
pass. These models are often customized in-house and information is presented
to users in the form of traffic light systems or other colour-coded user interfaces
(UIs). With specific reference to the TQA of machine-produced translations,
Doherty and colleagues (2013) noted an overwhelming preference in industry for
human assessment of MT (69 per cent) supplemented by AEMs (22 per cent) and
customized in-house measures (13 per cent).
Unsurprisingly, TQA in the industry is customer-focused. It has to be. Despite
this, Ray and colleagues (2013: 1) report that the industry’s own research ‘has
long shown that buyers and suppliers often disagree on what translation quality
means’. This lack of agreement on definition and measurement criteria leads to
mismatched expectations and conflict. They too identify a trend of TQA processes
being customized in-house, with ‘no generally accepted standards’, where ‘quality
expectations tend to follow prices up but not down’ (Ray et al. 2013: 2). Finally, in
terms of methodical rigour and subjectivity, Kelly and DePalma (2009: 1) describe
TQA in the industry as ‘often boiling down to the opinion of one person over that
of another’, with ‘too much time in arbitration mode’. They note that time and the
quality of the client’s linguistic assets are key factors in delivering high-quality
translation. However, they warn that such data can contain fundamental errors
even in spelling and punctuation, and they stress that ‘organizations are not testing
their evaluation tools to make sure that they are actually credible’ and that ‘inter-
rater reliability has fallen off the radar’ (Kelly and DePalma 2009: 7).
Psychometric principles
We have now seen similar issues in TQA across translation studies, MT and industry
contexts, and from different disciplinary and practitioner viewpoints. It is argued
here that common to each of these areas is the need to improve standards in TQA
in terms of: explicit definitions of quality; adherence to established conventions
138 Stephen Doherty
for testing validity and reliability, in line with other disciplines; greater awareness
of human factors in evaluation; and improved transparency in shared translation
and TQA data. In tackling each of these issues, we now look to psychometric
principles, and point towards examples of where improvements could be made
using examples of best practice and established guidelines that can be readily
adopted in translation studies, translation technology and the industry at large.
In a broad sense, psychometrics is the field of study pertaining to the theory and
practice of objective psychological measurement. Psychometric measurements are
used in a variety of social, behavioural and clinical sciences, and (more related to
the current context) are well established in language testing and psycholinguistics.
In each of these areas, psychometrics concerns itself with the construction and
empirical validation of assessment instruments, such as questionnaires, surveys
and tests, as well as of raters and test-takers themselves. In the softer sciences of
psychology and social science, psychometrics has an established history (Michell
1997), with seminal work establishing levels of measurement (Stevens 1946), and
a viewpoint that measurements in hard and soft sciences ‘are in no sense different’
(Reese 1943: 49).
While its stark behaviouralist perspective has proven difficult to apply to
some areas, it is argued here that TQA and its components, be they linguistic or
extralinguistic, stand to benefit from the incorporation of psychometrics much as
comparable disciplines in linguistics and language testing have done. The primary
framework underpinning psychometrics is classical test theory (for a detailed
introduction, see Traub 1997; Allen and Yen 2002; Hogan and Cannon 2007;
Gregory 2011), which champions validity and reliability – constructs that we will
now move on to discuss in the context of TQA.
Validity
Defined by the American Psychological Association (APA) as ‘the appropriateness,
meaningfulness and usefulness of the specific inferences made from test scores’
(cited in Bachman 1990: 243), the construct of validity raises the question of
whether the test is measuring what it is intended to (Cronbach 1971; Clifford
2001). Validity is divided into two aspects: internal validity and external validity.2
Internal validity relates to the content of the test, for example the items assessed
as part of a TQA model, linguistic features, n-grams, adequacy, fluency, etc.
External validity relates to the relationship between one’s ability as measured
on the test and one’s ability beyond the test. The ‘context’ of tests must be as
identical as possible to the real-world tasks (Weir 2005) – that is, the items being
assessed as part of TQA must be items that occur in translation in the real world,
rather than unrepresentative items, unrealistic text types, etc. Finally, to check the
validity of a test, Bachman and Palmer (1996) suggest following a logical analysis
that examines the clarity and appropriateness of the constructs included in the test.
It is here that the importance of operational definitions is once again apparent in
TQA, because of the needs to explicitly state what the tester wishes to test and to
avoid ambiguity, confusion and incorrect propagation of resultant findings.
To revisit the WMT examples noted earlier, a challenge to validity is the usage
of proxy measures in place of actual scores for a given variable. Several elements
comprise a WMT task: a translation task, an evaluation task and, more recently,
a quality estimation task. The 2012 WMT,3 for instance, measured ‘post-editing
effort’ by asking three evaluators to rate MT output on a scale of 1–5 in terms
of the extent of post-editing that they estimated was required to bring the MT
output under scrutiny up to publishable quality, rather than measuring actual post-
editing. A handful of researchers have attempted to address this problem by using
actual post-editing time (e.g. Specia et al. 2009); however, the use of such proxy
variables in TQA continues to be widespread.
Another example of proxy variables posing a threat to validity can be found in
the use of pupil fixations as a measure of cognitive effort for TQA (e.g. Doherty
et al. 2010) and of mouse movements as an indicator of attentional focus in
post-editing (e.g. Green et al. 2013), especially when it is well established that
translators rely primarily on keystrokes.4 Doherty and colleagues (2010) defend
this proxy variable by citing other literature that shows an established link
between fixations measured using eye-tracking and cognitive effort in linguistic
tasks, and note the impracticality of more direct observations, for example using
brain imaging. Green and colleagues (2013) make a similar case for mouse
movements, stating that they have been shown to correlate with attention, but
acknowledging that such observations come from studies of search-engine
usability – that is, simplistic tasks of an extremely limited linguistic nature that
are not actual translation or post-editing tasks. Mouse cursor movements may not
actually correspond to attentional focus in post-editing, however, thus potentially
affecting the validity of the study’s method and findings.
Reliability
The construct of reliability is used as a technical term to describe the amount
of consistency of test measurement in a given construct (Bachman 1990; Cohen
1994; Bachman and Palmer 1996; Clifford 2001). Reliability is measured by
examining the consistency of test scores. For example, if the same translation were
to be evaluated at different times, the same TQA criteria should yield consistently
similar, if not identical, results. As with validity, reliability can also be initially
tested by following a logical analysis as described in Bachman and Palmer (1996).
Following this, test–retest techniques can be used at fixed or random intervals to
ensure that reliability is maintained (Gipps 1994).
As an extension of reliability, when more than one evaluator (or ‘rater’) is
assessing the same translation, it is necessary to examine inter-rater reliability.
Ideally, there should be close similarity between the scoring that each evaluator
assigns to each translation – that is, they should be agreeing with each other.
Given the high degree of subjectivity widely acknowledged in TQA, this measure
is of particular interest and importance to us. Similar problems are faced in other
disciplines, in which there is also a danger of ‘rating by consensus’, whereby the
relationship (e.g. power, status, proficiency, motivation, intention, etc.) between
140 Stephen Doherty
Table 7.1 Thresholds for inter-rater reliability kappa scores
Value Category
0.01–0.20 Slight
0.21–0.40 Fair
0.41–0.60 Moderate
0.61–0.80 Substantial
0.81–1.00 Almost perfect
Source: Adapted from Landis and Koch (1977: 165)
evaluators may be such that one opinion has more sway than others. To avoid such
an issue, evaluations should be carried out independently and anonymously, where
possible. Where disputes between two evaluators arise, a third, neutral, evaluation
should be carried out by another evaluator and the averages of all scores used. If
one evaluator is found to be incorrect, the average of the two closest values can
be used as the final score (Fulcher 2003). Reliability is typically measured using
kappa scores (Cohen’s kappa or Fleiss’ kappa), as well as Pearson’s or Spearman’s
correlation coefficients, which are used to measure agreement between ‘pairs’ of
evaluators or test items.
To return to our WMT examples, Turchi and colleagues (2014) note widespread
inconsistency between evaluators in MT evaluation and quality estimation. On
closer examination of the reported inter-rater reliability scores for WMT (Bojar
et al. 2014) from 2011 to 2014 and across all 10 language pairs, one finds
averages from 0.260 (in 2013) to 0.395 (in 2011). Landis and Koch (1977) detail
the acceptable thresholds for kappa scores (see Table 7.1), which put WMT scores
in the ‘fair’ category – a result that leaves much room for improvement.
Evaluators
The essential aspect to any human evaluation is, of course, the evaluator, whose
skills and attributes must be suited to the evaluation task. An established means of
verifying this is to conduct a job task analysis (see, e.g., Koby and Melby 2013).
However, this can be problematic in settings in which limited resources often
make it difficult to hire professional translators to carry out evaluations, and a
detailed description of the evaluation task and its evaluators is typically seen as
an optional extra, rather than an essential prerequisite. Indeed, it is commonplace
not to see any description of evaluators whatsoever. It is also typically assumed
that students of languages or translation studies, or indeed anyone who possesses
some degree of proficiency in both languages or even only in the target language,
are appropriate evaluators. Very few studies actually provide detailed information
on evaluation and evaluators, let alone descriptions of guidelines and operational
definitions used in TQA processes. Added to this, our research community and its
publications do not insist that such information is provided.
To briefly test this observation, a sample of recent research papers from Machine
Translation was analysed. This flagship journal is highly respected and well cited,
and has continuously made significant contributions to both research and industry
communities.5 From 2010 through to 2014 (volumes 24–28, inclusive), there are
43 papers detailing empirical work containing TQA. Of these, 34 papers relied
exclusively on AEMs and only nine used human evaluation (although all TQA set-
ups in the 43 papers could have employed human evaluation). For the nine papers
describing human TQA, five provided information about the evaluators, including
professional experience, expertise, skills and language proficiency, while the other
four provided no information. In three of the same nine papers, evaluators were
already trained in TQA or were provided with brief training for the respective task
(e.g. post-editing), and details of the evaluation guidelines were provided in four.
Only one paper of the nine employed professional translators, while the remaining
eight described using students, untested native speakers and bilinguals, and self-
validating participants on online crowdsourcing platforms.
This example brings us to a second point: that of expertise and TQA-specific
training. Owing to limitations in project resources (including funding, time
and the ability to share data), there is, of course, a tendency for researchers to
ask students, fellow researchers and online communities to carry out TQA in
MT research. It may also be the case that the need for, and value of having,
professional translators who are trained as evaluators is not widely accepted in the
MT research community. While subjectivity and resources are issues that cannot
be ignored, with appropriate training, guidelines and larger sample sizes, many
of the known shortcomings of human evaluation can be easily addressed. But
even if human translators are used to evaluate MT output, significant knowledge
‘mismatches’ can arise between the research team and the evaluators. Lommel
et al. (2014: 36), reflecting on a collaborative TQA campaign between MT
researchers and professional translators, found that their training materials and
guides for evaluators ‘were insufficient to guide annotators when faced with
unfamiliar issues which they intuitively know how to fix but which they are not
used to classifying in an analytic scheme’.
Industry evaluators, on the other hand, typically do not have the option to recruit
students and researchers as evaluators; instead, translators, project managers
and those working within the organization with proficiency in both languages
may be evaluators, especially in community translation settings. For specialized
translation workflows, such as in medical and legal contexts, in which external
requirements mean that auditing trails must be kept, professional translators
with specific training in TQA should be used; however, it is not possible to find
independent research that details the extent and validity of this practice. Because
expert industry analysts such as Common Sense Advisory (see Ray et al. 2013)
continue to express extreme concern about the lack of reliability of industry TQA
processes, further action by all TQA stakeholders is evidently required.
Bias, too, can play a factor in TQA, for example professional, language,
cultural and universal cognitive biases, all of which are well documented in
other disciplines (see Dardenne and Leyens 1995; Maccoun 1998; Baron 2007).
For example, will students and researchers from translation studies be biased
in evaluating MT quality? And will MT researchers also be biased in the same
142 Stephen Doherty
evaluation? Coupled with this is the reliance on self-selective sampling and
self-description of participants who are not always the best judge of their own
language proficiency, and who (in the case of online crowdsourcing) may not
be truthful in describing their expertise and professional experience. All of
these possible subjective evaluator-based biases can compromise the validity
and reliability of TQA data, leading to new confounds and even confusion
for researchers trying to draw conclusions from their results, and then further
issues for their academic and industry readerships. Tighter controls on research
participants and evaluators in TQA are therefore essential, and should be
supplemented with psychometric testing, at the very least, as described briefly
in the previous sections.
Evaluation materials
Researchers also need to think carefully about the instructions provided to TQA
evaluators. Not only will this improve the validity, reliability and efficiency
of the evaluation, but also it will increase the value of the data gleaned from
the evaluation itself (e.g. to inform MT development, pricing, post-editing).
On examination of the literature, one will quickly find there are many different
definitions of the constructs in which researchers are interested: some are more
theoretical and fuzzy; others, more clearly defined for operationalization in the
confines of individual projects. Instructions and guidelines should be written
clearly and concisely, and should contain explicitly operationalized definitions
appropriate to the evaluator group and the TQA task. Assumptions cannot be
made about what undefined concepts, points on scales and arbitrary rankings
will mean to uninformed evaluators. If a definition is new to evaluators, or is a
variation of what they are accustomed to, it can be displayed on each evaluation
sheet for example, or above the error categories on a web-based TQA tool.
Evaluators should be given the opportunity to ask questions about the criteria
provided, and about the time and granularity requirements of the evaluation.
Time limits, and lack thereof, and whether they are known or unknown to
evaluators all have a considerable impact on evaluator performance. In all cases,
appropriate training, trial runs and testing are necessary to reduce the likelihood
of misunderstandings and poor performance, and to increase the probability of
higher inter-rater reliability scores being achieved – all of which can, of course,
make TQA more resource-efficient in the long term, since such measures reduce,
and even avoid, the need for repeating translation and evaluation tasks that have
not been performed correctly in the first place.
Presentation and context, too, are critical aspects that are often overlooked
in TQA. The impact of segmentation on text- and document-level attributes
such as cohesion can influence TQA scores both positively and negatively.
In MT research settings, TQA is typically conducted out of context and at
sentence level. In the industry, randomized samples are typically selected, the
size of which can vary from phrase to paragraph level, with CAT tools playing
an important role in their presentation. Both approaches are not without their
problems, in that evaluators must form their understanding and expectations of
the entire text on a sentence-by-sentence basis, but may wish to access the entire
text or to return to earlier sentences to modify scores based on new information
as they progress. Depending on time allocations and the presentation options
available in the evaluation platform or tool, such access and revisiting may not
be permitted. Finally, the presentation order of segments may be linear, as per
the structure of the entire text to be evaluated, or randomized. Each of these
ways of evaluating will, of course, influence evaluators and their scoring in
different ways.
Conclusion
In this chapter, we have addressed some of the main issues in TQA as viewed
from the perspectives of translation studies, MT and the translation industry.
In each case, we have identified areas in which closer adherence to existing
principles could greatly enhance the validity and reliability of TQA. We have also
made several recommendations and referred to many sources that could support
those involved in TQA in taking greater heed of a variety of subjective human
issues that arise in the evaluation of both human and machine translation. While
there is no ‘one size fits all’ solution or ‘correct’ answer, by considering possible
issues, consulting the literature, and being informed by established standards
and principles of experiment design, one can address possible confounds and
acknowledge limitations, and thereby achieve more valid and reliable results for
all TQA scenarios.
Notes
1 The discussion of human and automatic evaluation of MT output here is necessarily
brief and simplified. For more detailed treatments, see Denkowski and Lavie (2010)
and Koehn (2010). Note also that, further to the approaches mentioned in this chapter,
translation quality has also been measured by a range of other linguistic measures,
such as readability (e.g. Roturier 2006; Hartley et al. 2012) and comprehensibility
(e.g. Doherty 2012), as well as extralinguistic measures of usability (e.g. Doherty and
O’Brien 2012, 2014), acceptability (e.g. Flanagan 2009) and cognitive effort (e.g.
Doherty and O’Brien 2009; Doherty et al. 2010). Semi-automatic approaches have also
become increasingly popular for MT evaluation, which can take the form of data-driven
error analysis for diagnosing errors (e.g. Vilar et al. 2006) and using corpus tools (e.g.
Gaspari et al. 2014). Detailed discussion of these approaches is beyond the scope of this
chapter, however.
2 While there are other terminological variants commonly used for subcategories of
validity, the descriptions here are intended to avoid confusion and encourage introductory
comprehension. Further details of other subcategories can be found in Bachman (1990),
Bachman and Palmer (1996), Clifford (2001), Fulcher (2003), Weir (2005), and in the
guidelines detailed in the Standard for Educational and Psychological Testing (American
Educational Research Association et al. 2014).
3 See http://www.statmt.org/wmt12/ [accessed 12 March 2016].
4 In these cases, (increased) cognitive effort and (prolonged) attentional focus are in turn
interpreted as indicating a translation quality problem.
5 See http://link.springer.com/journal/10590 [accessed 12 March 2016].
144 Stephen Doherty
References
Allen, M., and W. Yen. 2002. Introduction to Measurement Theory. Prospect Heights, IL:
Waveland Press.
American Educational Research Association, American Psychological Association,
Joint Committee on Standards for Educational and Psychological Testing (U.S.),
National Council on Measurement in Education. 2014. Standards for Educational and
Psychological Testing. Washington, DC: American Educational Research Association.
Anckaert, P., J. Eyckmans and W. Segers. 2008. ‘Pour une évaluation normative de la
compétence de traduction’. ITL International Journal of Applied Linguistics 155:
53–76.
Bachman, L. 1990. Fundamental Considerations in Language Testing. Oxford: Oxford
University Press.
Bachman, L., and A. Palmer. 1996. Language Testing in Practice. Oxford: Oxford
University Press.
Baer, B., and T. Bystrova-McIntyre. 2009. ‘Assessing Cohesion: Developing Assessment
Tools on the Basis of Comparable Corpora’. In C. Angelelli and H. Jacobson (eds).
Testing and Assessment in Translation and Interpreting Studies: A Call for Dialogue
between Research and Practice. Amsterdam/Philadelphia: John Benjamins, 159–83.
Baker, M. 2011. In Other Words: A Coursebook on Translation. 2nd edn. London/
New York: Routledge.
Baron, J. 2007. Thinking and Deciding. 4th edn. New York: Cambridge University Press.
Bassnett-McGuire, S. 1991. Translation Studies. London/New York: Routledge.
Blatz, J., E. Fitzgerald, G. Foster, S. Gandrabur, C. Gouette, C. Kulesza, A. Sanchis and
N. Ueffing. 2003. Confidence Estimation for Machine Translation. Technical Report.
Baltimore, MD: Johns Hopkins University.
Bojar, O., C. Buck, C. Callison-Burch, C. Federmann, B. Haddow, P. Koehn, C. Monz,
M. Post, R. Soricut and L. Specia. 2013. ‘Findings of the 2013 Workshop on Statistical
Machine Translation’. In Proceedings of the 2013 Workshop on Statistical Machine
Bojar, O., C. Buck, C. Federmann, B. Haddow, P. Koehn, J. Leveling, C. Monz, P. Pecina,
M. Post, H. Saint-Amand, R. Soricut, L. Specia and A. Tamchyna. 2014. ‘Findings
of the 2014 Workshop on Statistical Machine Translation’. In Proceedings of the
Ninth Workshop on Statistical Machine Translation. Stroudsburg, PA: Association for
Computational Linguistics, 12–58.
Bowker, L. 2000. ‘A Corpus-Based Approach to Evaluating Student Translations’. The
Translator 6(2): 183–209.
Bowker, L. 2001. ‘Towards a Methodology for a Corpus-Based Approach to Translation
Evaluation’. Meta 46(2): 345–64.
Bowker, L. 2003. ‘Corpus-Based Applications for Translator Training: Exploring
the Possibilities’. In S. Granger, J. Lerot and S. Petch-Tyson (eds). Corpus-Based
Approaches to Contrastive Linguistics and Translation Studies. Amsterdam/New York:
Rodopi, 169–84.
Brunello, M. 2012. ‘Understanding the Composition of Parallel Corpora from the Web’.
In Proceedings of the Seventh Web as Corpus Workshop (WAC7), 7–13. Available at:
https://sigwac.org.uk/raw-attachment/wiki/WAC7/wac7-proc.pdf [accessed 12 March
2016].
Brunette, L. 2000. ‘Towards a Terminology for Translation Quality Assessment: A
Comparison of TQA Practices’. The Translator 6(2): 169–82.
Callison-Burch, C. 2009. ‘Fast, Cheap, and Creative: Evaluating Translation Quality
Using Amazon’s Mechanical Turk’. In Proceedings of the 2009 Conference on
Empirical Methods of Natural Language Processing. Stroudsburg, PA: Association for
Callison-Burch, C., P. Koehn, C. Monz, M. Post, R. Soricut and L. Specia, L. 2012.
‘Findings of the 2012 Workshop on Statistical Machine Translation’. In Proceedings of
the 2012 Workshop on Statistical Machine Translation. Stroudsburg, PA: Association
for Computational Linguistics, 10–51.
Callison-Burch, C., M. Osborne and P. Koehn. 2006. ‘Re-Evaluating the Role of BLEU in
Machine Translation Research’. In Proceedings of the 11th Conference of the European
Chapter of the Association for Computational Linguistics (EACL 2006). Stroudsburg,
Cao, D. 1996a. ‘Translation Testing: What Are We Measuring?’. In B. Lewandowska-
Tomaszcyzk and M. Thelen (eds). Translation and Meaning. Maastricht: Universitatire
Pers Maastricht, 525–32.
Cao, D. 1996b. ‘On Translation Language Competence’. Babel 42(4): 231–8.
Clifford, A. 2001. ‘Discourse Theory and Performance-Based Assessment: Two Tools for
Professional Interpreting’. Meta 46(2): 365–78.
Cohen, A. 1994. Assessing Language Ability in the Classroom. 2nd edn. Boston, MA:
Heinle & Heinle.
Cronbach, L. 1971. ‘Test Validation’. In R. Thorndike (ed.). Educational Measurement.
2nd edn. Washington, DC: American Council on Education, 443–507.
Dardenne, B., and J.-P. Leyens. 1995. ‘Confirmation Bias as a Social Skill’. Personality
and Social Psychology Bulletin 21(11): 1229–39.
Denkowski, M., and A. Lavie. 2010. ‘Choosing the Right Evaluation for Machine
Translation: An Examination of Annotator and Automatic Metric Performance on
Human Judgment Tasks’. Paper presented at the Ninth Conference of the Association
for Machine Translation in the Americas, Denver, CO, 31 October–4 November.
Doherty, S. 2012. Investigating the Effects of Controlled Language on the Reading and
Comprehension of Machine Translation Texts: A Mixed-Methods Approach. PhD thesis.
Dublin City University. Available at: http://doras.dcu.ie/16805/ [accessed 12 March 2016].
Doherty, S. 2016. ‘The Impact of Translation Technologies on the Process and Product of
Translation’. International Journal of Communication 10: 1–23.
Doherty, S., and S. O’Brien. 2009. ‘Can MT Output Be Evaluated through Eye Tracking?’
In Proceedings of Machine Translation Summit XII. Ottawa, ON: International
Association for Machine Translation, 214–21.
Doherty, S., and S. O’Brien. 2012. ‘A User-Based Usability Assessment of Raw Machine
Translated Technical Instructions’. In Proceedings of the 10th Conference of the
Association for Machine Translation in the Americas (AMTA). Stroudsburg, PA:
AMTA, 28–31.
Doherty, S., and S. O’Brien. 2014. ‘Assessing the Usability of Raw Machine Translated
Output: A User-Centred Study using Eye Tracking’. International Journal of Human
Computer Interaction 30(1): 40–51.
Doherty, S., F. Gaspari, D. Groves and J. van Genabith. 2013. ‘Mapping the Industry I: Findings
on Translation Technologies and Quality Assessment’. QTLaunchPad. June. Available
at: https://www.researchgate.net/publication/257227488_Mapping_the_Industry_I_
Findings_on_Translation_Technologies_and_Quality_Assessment_Mapping_the_
Industry_I_Findings_on_Translation_Technologies_and_Quality_Assessment [accessed
12 March 2016].
146 Stephen Doherty
Doherty, S., S. O’Brien and M. Carl. 2010. ‘Eye Tracking as an MT Evaluation Technique’.
Machine Translation 24(1): 1–13.
Eyckmans, J., P. Anckaert and W. Segers. 2009. ‘The Perks of Norm-Referenced
Translation Evaluation’. In C. Angelelli and H. Jacobson (eds). Testing and Assessment
in Translation and Interpreting Studies: A Call for Dialogue between Research and
Practice. Amsterdam/Philadelphia: John Benjamins, 73–93.
Federmann, C. 2010. ‘Appraise: An Open-Source Toolkit for Manual Phrase-Based
Evaluation of Translations’. In Proceedings of the Seventh Conference on International
Language Resources and Evaluation (LREC2010). Paris: European Language
Resources Association, 1731–4.
Flanagan, M. 2009. ‘Using Example-Based Machine Translation to Translate DVD
Subtitles’. In Proceedings of the Third Workshop on Statistical Machine Translation.
Fulcher, G. 2003. Testing Second-Language Speakers. London: Pierson Longman Press.
Gaspari, F., H. Almaghout and S. Doherty. 2014. ‘A Survey of Machine Translation
Competencies: Insights for Translation Technology Educators and Practitioners’.
Perspectives: Studies in Translatology 23(3): 333–58.
Gipps, C. 1994. Beyond Testing: Towards a Theory of Educational Assessment. London:
Falmer Press.
Graham, Y., T. Baldwin, A. Moffat and J. Zobel. 2013. ‘Continuous Measurement
Scales in Human Evaluation of Machine Translation’. In Proceedings of the Seventh
Linguistic Annotation Workshop and Interoperability with Discourse. Stroudsburg, PA:
Association for Computational Linguistics, 33–41.
Green, S., J. Heer and C. Manning. 2013. ‘The Efficacy of Human Post-Editing for
Language Translation’. In Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems (CHI’13). New York: Association for Computing Machinery,
439-48.
Gregory, R. 2011. Psychological Testing: History, Principles, and Applications. 6th edn.
Boston, MA: Allyn & Bacon.
Hartley, A., M. Tatsumi, H. Ishara, K. Kageura and R. Miyata. 2012. ‘Readability and
Translatability Judgments for Controlled Japanese’. In Proceedings of the 16th
Conference of the European Association for Machine Translation. Trento: Fondazione
Bruno Kessler, 237–44.
Hatim, B., and I. Mason. 1990. Discourse and the Translator. London: Longman.
Hatim, B., and I. Mason. 1997. The Translator as Communicator. London/New York:
Routledge.
Hogan, T., and B. Cannon. 2007. Psychological Testing: A Practical Introduction. 2nd
edn. Hoboken, NJ: Wiley & Sons.
Hönig, H. 1998. ‘Positions, Power and Practice: Functionalist Approaches and Translation
Quality Assessment’. In C. Schaeffer (ed). Translation and Quality. Clevedon:
Multilingual Matters, 6–34.
House, J. 1997. Translation Quality Assessment. Tübingen: Gunter Narr Verlag.
House, J. 2009. ‘Quality’. In M. Baker (ed.). Routledge Encyclopaedia of Translation
Studies. London/New York: Routledge, 222–5.
Kelly, N., and D. DePalma. 2009. The Case for Terminology Management. Boston, MA:
Common Sense Advisory.
Kenny, D. 2011. ‘The Ethics of Machine Translation’. In Proceedings of the 11th New
Zealand Society of Translators and Interpreters Annual Conference 2011. Auckland:
NZSTI, 121–31.
Koby, G., and A. Melby. 2013. ‘Certification and Job Task Analysis (JTA): Establishing
Validity of Translation Certification Examinations’. International Journal for
Translation and Interpreting Research 5(1): 174–210.
Koehn, P. 2010. Statistical Machine Translation. Cambridge: Cambridge University Press.
Koehn, P., and Monz, C. 2006. ‘Manual and Automatic Evaluation of Machine Translation
between European Languages’. In Proceedings of the 2006 Workshop on Statistical Machine
Koponen, M. 2012. ‘Comparing Human Perceptions of Post-Editing Effort with Post-
Editing Operations’. In Proceedings of the Seventh Workshop on Statistical Machine
Labaka, G., C. Espana-Bonet, L. Marquez and K. Sarasola. 2014. ‘A Hybrid Machine
Translation Architecture Guided by Syntax’. Machine Translation 28(2): 91–125.
Landis, J. R., and G. Koch. 1977. ‘The Measurement of Observer Agreement for Categorical
Data’. Biometrics 33(1): 159–74.
Lommel, A., A. Burchardt, M. Popovic, K. Harris, E. Avramidis and H. Uszkoreit. 2014.
‘Using a New Analytic Measure for the Annotation and Analysis of MT Errors on Real
Data’. In Proceedings of the 17th Conference of the European Association for Machine
Translation. Trento: Fondazione Bruno Kessler, 165–72.
Maccoun, R. 1998. ‘Biases in the Interpretation and Use of Research Results’. Annual
Review of Psychology 49(1): 259–87.
Michell, J. 1997. ‘Quantitative Science and the Definition of Measurement in Psychology’.
British Journal of Psychology 88(3): 253–83.
Nord, C. 1991. ‘Scopos, Loyalty and Translational Conventions’. Target 3(1): 91–109.
Papineni, K., S. Roukos, T. Ward and W. Zhu. 2002. ‘BLEU: A Method for Automatic
Evaluation of Machine Translation’. In Proceedings of the 40th Annual Meeting of
the Association for Computational Linguistics. Stroudsburg, PA: Association for
Ray, R., D. DePalma and H. Pielmeier. 2013. The Price–Quality Link. Boston, MA:
Common Sense Advisory.
Reese, T. 1943. ‘The Application of the Theory of Physical Measurement to the
Measurement of Psychological Magnitudes with Three Experimental Examples’.
Psychological Monographs 55(3): 1–89.
Reiss, K. 2000. Translation Criticism: The Potentials and Limitations: Categories and
Criteria for Translation Quality Assessment. Manchester: St Jerome.
Roturier, J. 2006. An Investigation into the Impact of Controlled English Rules on the
Comprehensibility, Usefulness, and Acceptability of Machine-Translated Technical
Documentation for French and German Users. PhD thesis. Dublin City University.
Available at: http://doras.dcu.ie/18190/1/Johann_Roturier_20130116094552.pdf
Sager, J. 1989. ‘Quality and Standards: The Evaluation of Translations’. In C. Picken (ed.).
The Translator’s Handbook. 2nd edn. London: ASLIB, 91–102.
Schäffner, C. (ed.). 1998. Translation and Quality. Philadelphia, PA: Multilingual Matters.
Snell-Hornby, M. 1992. ‘The Professional Translator of Tomorrow: Language Specialist
of All-Round Expert?’. In C. Dollerup and A. Loddegaard (eds). Teaching Translation
and Interpreting: Training, Talent and Experience. Amsterdam: John Benjamins, 9–22.
Snover, M., B. Dorr, R. Schwartz, L. Micciulla and J. Makhoul. 2006. ‘A Study of
Translation Edit Rate with Targeted Human Annotation’. In Proceedings of Seventh
Conference of the Association for Machine Translation in the Americas. Stroudsburg,
PA: AMTA, 223–31.
148 Stephen Doherty
Snow, R., B. O’Connor, D. Jurafsky and A. Ng. 2008. ‘Cheap and Fast: But Is It Good?
Evaluating Non-Expert Annotations for Natural Language Tasks’. In Proceedings of the
2008 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,
Specia, L., and A. Farzindar. 2010. ‘Estimating Machine Translation Post-Editing Effort
with HTER’. In Proceedings of the Second Joint EM+/CNGL Workshop Bringing MT
to the User: Research on Integrating MT in the Translation Industry, 33–41. Available
at: https://amta2010.amtaweb.org/AMTA/papers/7-02-00-VentsislavZhechevEd.pdf
Specia, L., M. Turchi, N. Cancedda, L. Micciulla and J. Makhoul. 2009. ‘Estimating the
Sentence-Level Quality of Machine Translation Systems’. In Proceedings of the 13th
Annual Conference of the European Association of Machine Translation. Trento:
Fondazione Bruno Kessler, 28–35.
Stevens, S. 1946. ‘On the Theory of Scales of Measurement’. Science 103(2684): 677–80.
Tatsumi, M. 2009. ‘Correlations between Automatic Evaluation Metric Scores, Post-
Editing Speed, and Some Other Factors’. In Proceedings of the Twelfth Machine
Translation Summit. Ottawa, ON: International Association for Machine Translation,
332–9.
Tatsumi, M., and J. Roturier. 2010. ‘Source Text Characteristics and Technical and
Temporal Post-Editing Effort: What Is Their Relationship?’ In Proceedings of the
Second Joint EM+/CNGL Workshop Bringing MT to the User: Research on Integrating
MT in the Translation Industry, 43–51. Available at: https://amta2010.amtaweb.org/
AMTA/papers/7-02-00-VentsislavZhechevEd.pdf [accessed 12 March 2016].
Traub, R. 1997. ‘Classical Test Theory in Historical Perspective’. Educational
Measurement: Issues and Practice 16(4): 8–14.
Turchi, M., M. Negri and M. Federico. 2014. ‘Data-Driven Annotation of Binary MT
Quality Estimation Corpora Based on Human Post-Editions’. Machine Translation
28(3–4): 281–308.
Turian, J., L. Shen and I. Melamed. 2003. ‘Evaluation of Machine Translation and its
Evaluation’. In Proceedings of the Ninth Machine Translation Summit (MT Summit IX).
Ottawa, ON: International Association for Machine Translation, 386–393.
Uzar, R. 2004. ‘A Toolbox for Translation Quality Assessment’. In B. Lewandowska-
Tomaszczyk (ed.). Practical Applications in Language and Computers, Vol. 9.
Frankfurt: Peter Lang, 155–66.
Vilar, D., J. Xu, F. D’Haro and H. Ney. 2006. ‘Error Analysis of Statistical Machine
Translation Output’. In Proceedings of the Fifth International Conference on Language
Resources and Evaluation (LREC2016). Paris: European Language Resources
Association, 697–702.
Waddington, C. 2004. ‘Should Student Translations Be Assessed Holistically or through
Error Analysis?’ Lebende Sprachen 49(1): 28–35.
Weir, C. 2005. Language Testing and Validation: An Evidence-Based Approach. New York:
Palgrave Macmillan.
Williams, M. 2001. ‘The Application of Augmentation Theory to Translation Quality
Assessment’. Meta 46(2): 326–44.
8 Cn U read ths?
The reception of txt language in subtitling
Alina Secară
Introduction
This chapter investigates the use and reception in subtitling of non-standard txt
spellings. The research is motivated by the belief that, in interlingual subtitling –
a physically constrained type of translation, in which the average viewer spends
more than half of the subtitle display time ‘within an area . . . less than about 13
per cent of the movie scene’ (Goldstein et al. 2007: 960) – it is worth trying out
shorter txt forms, which could potentially enable viewers to distribute their visual
resources across more of the screen. The study is also motivated by the facts
that communication is increasingly mediated by technology and that translation
consumption patterns have changed with increased publication of translated
products on the Internet. We thus argue that translators and subtitlers have to
know how to accurately respond to these changes if they are to create adequate
products for their target audiences.
Our interest in exploring this theme also stems from a desire to challenge a
certain uniformity observed in subtitle creation. We contend that media market
transformations should also trigger changes in the way in which subtitles are
created and presented on different platforms, and for different audiences. Most
subtitles produced professionally around the world today follow very similar
practices and techniques, despite the ever-changing nature of audiovisual genres
and the diversification in their mediums of distribution. Whether in cinemas,
on DVDs or on the Internet, interlingual subtitles display very few alterations
in style, linguistic form and overall presentation. The fragmentation of media
consumption has been accompanied by surprisingly few changes in the subtitling
process. This poses not only the question of why subtitles continue to be created
in a uniform manner across media and genres, but also, more significantly, how
would consumers react and behave if an alternative were offered?
In offering an alternative approach to professional subtitling for specific
new distribution contexts, we draw inspiration from non-traditional translation
practices, including fan translation and crowdsourced translation. Members of
these communities pioneered an unconstrained approach to subtitling, in which
language creativity and imaginative techniques for the presentation of the
translated content were employed to successfully deliver content on the Internet
(Pérez González 2006). In the same spirit, this study analyses the effect that the
150 Alina Secară
use in professionally created subtitles of non-traditional subtitling practices –
more specifically, the use of txt language, with its shortened forms such as C U,
gr8 and thx – has on a specific Internet audience.
In this context, questions arise as to how viewers perceive such non-traditional
subtitles and whether, and if so, how, their reading of what we will call creative
subtitles containing txt language differs from their reading of traditional subtitles.
This will represent the main research question here. We are also interested in
whether our viewers’ reading of these more creative subtitles changes with
exposure and, lastly, in how open consumers are to non-standard approaches.
We thus designed an experiment to enable us to draw comparisons between
a situation in which viewers are shown traditional interlingual subtitles and a
situation in which subtitles containing creative language were used. The cognitive
effort of viewing subtitles was measured both quantitatively, using indicators
of processing difficulty recorded by an eye-tracker, and qualitatively, through
a questionnaire. The rationale for the study, as well as its methodology and
results, are presented in this chapter, following a preliminary examination of the
phenomenon of txt language.
What is txt language?

Having disturbed the established conventions of standard English spelling, informal
interactive written discourse in txt lingo has become associated with eccentricity,
exotic choice, youth culture, idiosyncrasies and, in some people’s opinion,
incompetence (Humphrys 2007). However, the sheer volume of txt language and
instant messaging (IM) language being used today makes it impossible to dismiss
this phenomenon as substandard. In 2013, 92 per cent of the population in the
United Kingdom (UK) owned a mobile phone and the proportion of adults with a
smartphone was 51 per cent (Ofcom 2013). Moreover, the same source reported
the text message to be the most commonly used method for communicating with
family and friends, the average adult in the UK sending at least one message a day
(Ofcom 2013). In Australia, a 2006 online survey revealed that respondents (50
per cent aged 25 or younger) sent, on average, nine text messages per day (James
2007, cited in Kemp 2010). For the reasons to be outlined shortly, it is likely that
many of these messages contained txt lingo.
Producing and reading messages is therefore a very popular activity, but it is
one that was originally developed within the confines of a 160-character message
limit, with texts often displayed on a small screen. Because speed has always
been of the essence in this context and because early handsets involved a rather
laborious process for creating a message, abbreviations and shortened forms
were adopted from computer-mediated communication (CMC), in which they
had been in circulation for several years (Kemp 2010). Advances in technology,
such as the introduction of predictive texting, bigger phone screens, the virtual
elimination of the 160-character limit by automatic concatenation of messages
and the falling cost of sending text messages, have eliminated most of the
original constraints and therefore could have led to the disappearance of textisms
Cn U read ths? 151
in this medium. Yet not only have textisms not disappeared, but also they have
continued to grow and diversify (Baron 2010b).
While not homogenous, vernacular orthographies present sensible variations from
standard ones (see Shortis 2007), showing a high degree of linguistic convention
and logic. Even if there is no standard rule as to how a particular word is spelt,
members of a group do influence each other in the creation and implementation of
new forms. While linguistic txt characteristics such as vowel deletion, acronyms,
abbreviations and clipping are known and listed in many research articles such as
those discussed in what follows, sensible variations still exist arising from different
contextual realizations of such writing and writers’ personal styles. Typical texting
behaviour does not include as many juxtapositions of idiosyncratic or obscure usages
as one might imagine. As a matter of fact, empirical studies show that txt language is
more conservative than previously claimed (Tagliamonte and Denis 2008). Crystal
(2009) and Baron (2010a) both arrive at the conclusion that electronically mediated
communication has had little impact on vocabulary, with hardly any novelties at the
grammatical level, and that the only area in which ‘noticeable innovation is taking
place is orthography’ (Crystal 2009: 95).
Shortis (2007: 7–8) lists the special features of English txt language orthography,
breaking them into three categories:
• features used for reasons of economy, including omission of vowels (good

 gd), letter and number homophones (are  r), consonant reduction
(immediately  imedtly);
• devices used to simulate spoken languages, such as accent simulation (going
 goin), reduplication for stretched sounds for emphasis (Soooooooo) and
capitals to indicate paralinguistic details such as tonality; and
• graphical and kinaesthetic devices, such as emoticons, colours, movement
and alphabetical constructions.
Interestingly, creative uses of language in IM and fan translations are not

markedly influenced by user age; rather, it is the context that triggers various uses
of language. This is also supported by Crystal (2006, 2009), who carries out an
extensive study into what he calls ‘hi-tech lingo’. Central to his study is the idea
that these practices are not new and that similar creative features have been in
use since at least the seventeenth century. He identifies six distinctive features of
textese or ‘hybrid shorthand’ – namely, pictograms and logograms (L, @, zzz),
initialisms (OMG, NP), omitted letters (frm, wat, wuld), non-standard spellings
(thru, cos, fone), shortenings (bro, min, mo) and genuine novelties (cu2nite, B4).
While txt language is still considered deficient and granted low status, texts
in highly visible domains – such as advertising – are starting to show timid use
of textisms. Some suggest that ‘the convenience of email, IM and texting tempts
us to sacrifice intellect and elegance for immediacy’ (Baron 2010b: 232). While
this may be true to some extent, some contexts impose physical and temporal
constraints, and may not require elegant style. Moreover, an increasing number
of studies show positive correlations between the use of non-standard spelling
152 Alina Secară
variations and different types of linguistic proficiency (e.g. literacy, spelling and
reading scores) (e.g. Crystal 2006; Coventry University 2009; Plester et al. 2009;
Kemp 2010).
Interestingly, as people of different ages and backgrounds gain access and
confidence in using the Internet, the profile of the txt lingo user also changes.
A recent study including participants with ages ranging from 18 to 50, and
investigating the sex and age differences in the use of emoticons in Internet chat
rooms, concluded that sex and age do not play a role in the uptake and variety
of emotions expressed (Fulwood et al. 2013). Moreover, although women used
emoticons more frequently, there was no difference between men and women
in terms of the range of emotions expressed. This suggests that, with increased
production and exposure to txt language practices, individuals may increasingly
be seen to more readily accept this currently non-conformist practice.
Rationale for using txt lingo in subtitling

Similarly to IM and mobile texting platforms where it has been employed for many
years, the use of short txt lingo or ‘squeeze text’ (Carrington 2004) in subtitling
can be motivated by three principles, as described by Werry (1996, cited in Shortis
2007). First, there are features of space economy and text entry reduction that
govern all three of these contexts. Strict space limitations mean most subtitling
environments can only allow up to 39 characters per line, Short Message Service
(SMS) texts impose a limit of 160 characters and IM environments, even if not as
strictly conditioned, seem in practice to average around 30 characters per line (Ling
and Baron 2007). Although some of these limitations have been mitigated by, for
example, improved display on smartphones, the issue of physical constraints is
still central to the production of content in these environments. Moreover, the
appearance of newer platforms, such as Twitter, which revolve around the idea
of transmitting a message as quickly and economically as possible, makes our
investigation also applicable to newer contexts.
Second, as a powerful tool for depicting orality features, txt language is
frequently used to generate content that can be labelled as written speech. Spelling
by simulating spoken language has already been explored for the translation of
dialects in multilingual subtitling and the transfer or enhancement of orality
features in monolingual subtitling (Zarate and Eliahoo 2014).
Last, but not least, all three environments feature a shift to multimodal visual
and graphical effects and iconicity in which the linguistic sign is pushed to the
periphery of meaning making. Traditionally, subtitling has denied non-verbal
modalities, packaging and presenting exclusively verbal information. Nevertheless,
one aspect – namely, the use of emoticons – has already been proposed and used
with reasonable success in monolingual subtitling (Civera 2005).
Having addressed the rationale for inclusion of txt lingo in professional subtitling
for certain Internet media and audiences, let us briefly consider a few translation
contexts in which these practices are already in use – namely, social translation
platforms. As discourse communities, fan communities create and adopt discourse
Cn U read ths? 153
features that best serve their interests. In her translation, a fansubber will adopt
those linguistic specificities that she feels best describe the idiosyncrasies of the
film translated (O’Hagan 2008). In O’Hagan’s study, the most salient feature
differentiating a fansubber from a professional translator was the use of orthographic
devices to recreate the stereotypical profile of a source manga character. What is
emphasized by this study is the naturalness of these techniques in an environment
that has a powerful linguistic identity. A tentative link between fan subtitles and
creative spellings has been identified in the works of Nornes (1999) and Howell
(2006). Howell (2006: 304) argues, with reference to Japanese anime, that the
technical constraints that subtitling imposes on stylistic functions have sometimes
been overstated and that, ‘depending on the type of subtitling adopted, the narrative
function of characterization in the Japanese dialogue can be reproduced by the use
of a variety of compensatory procedures’. To this end, he analyses two subtitle files
produced for Grave of the Fireflies in which pronunciation spellings, for example
the repetition of a vowel to mark emphasis, are preferred by the subtitler to recreate
the colloquial style. Called ‘abusive’ practices, they are fully analysed by Nornes
(1999), who describes at length examples such as translator Yamamoto Masashi
using @~?!# to translate swearwords.
We therefore believe that new orthographic devices are employed not only to
attract attention to a particular word, but also to overcome the limitations of the
traditional writing palette by adapting it to meet new challenges.
Methodology
To test the effect that the introduction of txt lingo in subtitles would have on the
viewing experience, as well as to analyse viewers’ attitudes to this technique,
two experimental groups were set up. One group, C2, was asked to read
subtitles in the treatment condition (containing some words written in English
txt lingo), while the other group, C1, was presented with the same subtitles, but
containing standard English spelling – that is, in the non-treatment condition.
The comparative analysis was based on eye-tracking data, combined with a
questionnaire designed to elicit data on the comprehension and tolerance of new
practices using txt forms.
The stimulus was an extract from a 2009 French documentary Surfeurs du
Paradis (Jacquemin and Pinson 2009). This source material was selected because
the documentary targeted viewers belonging to an age group similar to that of
our intended target audience – namely, technology-savvy 20–40-year-olds.
Moreover, the subject of the documentary – surfing championships – was also
believed to be of interest to this group. The main experiment also included a
short 2-minute ‘warm-up video’, from the same documentary, for which gaze
data was not recorded. This allowed participants in both groups to familiarize
themselves with the environment. The warm-up clip and its respective subtitles
were consistent with the experiment clip. It included 20 subtitles, both one-line
and two-line ones, using the same presentation and timing as the experiment
subtitles. The experiment clip started immediately after the warm-up one.
154 Alina Secară
Two English subtitle tracks containing 136 subtitles each were prepared for
the experiment clip, one for C1 and one for C2, following identical technical
parameters: the same reading speed of 720 characters per minute, a maximum of
39 characters per line, the same centred justification and the same rules regarding
timing over shot changes. Moreover, the subtitle onset corresponded to the voice
onset, and the two subtitling tracks shared the same timing codes and presentation,
as well as the same number of one-line and two-line subtitles. The only difference
between the two subtitling files was that C2 contained txt spellings for a number
of words, the rest following traditional spelling similar to C1.
The number of changes introduced may seem very timid (Table 8.1), but they
mirror the frequency of txt forms in specially collected corpora. Baron (2010b)
gathered a 12,000-word corpus of IM speak and found only 90 acronyms, with a
significant number of forms occurring only once. One of the most frequent forms,
lol, occurred only 76 times. Tagliamonte and Denis (2008), in a much bigger
corpus, reported that txt forms represented only 2.5 per cent of the total corpus
word count. Frequent forms, such as omg and u, featured more prominently in the
corpus, but other known forms, such as wtf and brb, accounted for only 0.02 per
cent and 0.04 per cent of the corpus, respectively.
Moreover, statistically, the difference in the number of characters between the
two files was significant (Table 8.2). The Shapiro–Wilk test of normality indicates
normally distributed data (W (147) = 0.20, p > 0.05). Following this result, we
undertook an independent sample t-test, which shows that C1 (M = 44.9116,
SE = 1.3224) contained a significantly higher number of characters than C2
(M = 39.4762, SE = 1.1533) (t (147) = 3.098, p < 0.05, r = 0.17). Therefore
the files included in the two conditions were significantly different in terms of
number of characters that they presented to viewers for reading.
There is a series of factors, such as word length, word frequency, linguistic
complexity of the text and text presentation, among others, which can have an
impact on the distribution of attention and the gaze-path trajectory during reading
(Caffrey 2009). It was therefore important to make sure that the two subtitling
Table 8.1 Word and character count in subtitling files for C1 and C2
Condition Clip duration No. of No. of No. of No. of characters

subtitles words characters (no spaces)
C1 14 mins 33 secs 136 1,177 6,254 5,320

C2 14 mins 33 secs 136 1,173 5,485 4,555
Table 8.2 Reduction in character count in C2
C2 Reduction (%)
Word count 0.2

Character count 12.3
Character count (no spaces) 14.4
Cn U read ths? 155
tracks for the two experiment conditions were as similar as possible in terms
of presentation and comparable in terms of difficulty. The source clip and the
standard English subtitles did not contain any technical or exotic terms, and this
was tested using two parameters – namely, word frequency and the Flesch reading
ease index, which is a general index for English language in use. The overall
Flesch value for the C1 subtitle file was 84.4. The first half of the text achieved
87.1 and the second, 82.6. Moreover, the word-frequency analysis generated using
Web Vocabprofile (Cobb 2008) showed that 81.91 per cent of the vocabulary in
the text belonged to the 1,000 most frequent words in English and that the lexical
density (content words/total) was 0.49. Therefore the C1 text is seen as ‘easy to
read’, and only a small variation in terms of difficulty exists between the first and
the second part of the file.
It was therefore essential that our txt subtitles match this level of familiarity
and using corresponding high-frequency txt forms was essential. We based our
selection of txt variations to be included in the creative subtitle C2 track on a
corpus of 190,000 SMS messages (Tagg 2009). More specifically, the majority
of the txt forms included came from the top 150 creative SMS spellings in this
corpus and were therefore sufficiently frequent in this context. When not from
the Tagg corpus, the txt forms included in the subtitling file were created using
the Transl8it!1 platform. Moreover, because no large publicly accessible txt lingo
corpus exists, relative frequencies were also obtained from ukWaC, a very large
web-derived corpus and ‘one of the largest freely available linguistic resources
for English’ (Ferraresi et al. 2008: 1), which also contains a significant number
of txt forms. Of the total of 125 unique txt forms used in the treatment condition
subtitling file, 61 are listed in ukWaC. Moreover, of the total 1,914,150,196
ukWaC tokens, the 61 txt forms included in the experiment C2 file represent
almost 0.25 per cent (4,732,679 tokens). Given that these 61 types make up only
0.0016 per cent of the total 3,798,106 corpus types, their overall token number in
the ukWaC is impressive and it remains the best available approximation of the
frequencies of the experiment’s txt forms on the web.
That said, variability is widely accepted among texters and also well
documented. In Kemp’s (2010: 61) study, for example, ‘out of 87 word types
abbreviated by 2 or more people, only 17 were abbreviated in a consistent way’,
and these were very common – usually one-syllable letter/number homophones,
such as r, 4, u, c. This is the reason why some examples of textese included in our
file were not listed in the ukWaC or Tagg corpora, but naturally appeared when
applying txt language to the original text.
To account for the increase in difficulty in the second part of the C1 file,
the C2 file also needed to progressively feature more complex txt lingo. This
was also crucial because we also wanted to assess the effect, if any, of exposure
to txt language in a subtitling environment. Therefore the difference in lexical
txt diversity between the first and the second part of the subtitling file in the
treatment condition was measured by gathering general quantitative data and
calculating type token ratios (TTR), where a higher TTR indicates less repetition
of vocabulary for samples of roughly the same length (see Table 8.3).
156 Alina Secară
Table 8.3 txt lingo characteristics in the first and second parts of the C2 subtitling file
First 68 subtitles in C2 Last 68 subtitles in C2
txt lingo tokens 194 221

txt lingo types 64 101
txt lingo types in ukWaC 27 50
Type–token ratio (TTR) 0.329 0.457
Figure 8.1 Example of subtitles in C1, first and second parts of the file
Moreover, the txt lingo techniques applied in the creation of the new forms
vary more in the second part. If the txt forms included in the first 68 subtitles of
the file were generally created by using omitted letters, non-standard spellings
and logograms, the last 68 subtitles included, in addition to these, initialisms and
genuine novelties. These combine two or more techniques to create the final txt
forms, such as gr8 and aw8. The incidence of txt modifications in the second
part of the file is higher than in the first part. Not all word forms underwent txt
modifications, however, because the aim was to emulate normal txt language
behaviour. Figures 8.1 and 8.2 are indicative of the changes made.
To extract eye-tracking data, the subtitles for both videos were first marked
individually in the eye-tracker analysis module as crude areas of interest (AOIs).
This involved the drawing of a box around each individual subtitle, covering its
width and height, and the inclusion of a border around the subtitle of approximately
two characters on each side. This is a frequent practice, also reported in other eye-
tracking studies (e.g. Kruger and Stein 2013), and it ensures that all fixations
belonging to a subtitle, even if slightly outside the word area, are analysed.
Out of a total of 20, eight participants with a mean age of 23 (SD = 3.31) were
included in the analysis. Level of education, age and gender were elicited as part
of a screening questionnaire, as well as familiarity with subtitles, general visual
acuity and familiarity with subtitling. Respondents who did not have English as
their mother tongue or were not between the ages of 18 and 30 were automatically
excluded. Those who had some knowledge of French were also excluded. Some
were excluded as a result of poor eye-tracking data, the threshold for measuring
data quality being set at 85 per cent, following close inspection of the recordings
Cn U read ths? 157
Figure 8.2 Example of subtitles in C2, first and second parts of the file
and discussions with Tobii developers (see below). Equal group and gender
distribution in the two groups was also achieved – that is, four males and four
females. Ethics approval was obtained from the University of Leeds Research
Ethics Committee and the experiment briefing followed standard ethics guidelines.
It also stressed that the purpose was to analyse how reading of subtitles was done
under certain experimental conditions and not to assess the quality or efficiency
of each participant’s reading. The participants also received a financial incentive
to participate in the current study. In terms of their txt language competence, all
four participants included in the treatment condition (txt language) reported being
very comfortable (the highest on a four-point scale) when reading the text in the
four txt-specific questions in the screening questionnaire and also indicated that
they were consumers of txt language (three out of four consumed txt language
daily and one weekly).
Tobii X120 – a stand-alone eye-tracking unit, with a 50 Hz sampling rate,
allowing unrestrained head motion – was the technology used. The experiment
took place in a soundproof lab with no natural light and the video stimuli were
shown on a 19-inch monitor at a viewing distance of approximately 60 cm. A
five-point calibration was conducted at the beginning of the experiment.
Regressions, fixation counts and mean fixation durations

In this chapter, regressions are the principal eye-tracking metric used to test
potential differences in the processing of subtitles by the two groups. Regressions
are ‘re-readings of the same item, string, sentence or subtitle, and proceed along
a horizontal axis’ (Ghia 2012: 80). They are generally accepted to represent
comprehension and processing difficulty (Rayner and Pollatsek 1989; Pollatsek
and Rayner 2006; Hefer 2010), because they signal disruptions in a normal
reading process.
More specifically, regressive eye movements have been shown to be indicative
of vocabulary and syntactic difficulties. Summarizing the finding of a 2003 text
158 Alina Secară
reading study by Morris and Williams, Hefer (2010: 22) talks about connections
that readers make between the informative context and unfamiliar words, with the
effect that context leads readers to ‘make regressions to a preceding unfamiliar
word. More regressions were found when the preceding word was unfamiliar than
for familiar word(s)’. Pollatsek and Rayner (2006) also show that readers not only
make longer fixations and shorter saccades when text is more difficult to read and
comprehend, but also use more regressions. Rayner and colleagues (2006) show
that, in addition to leading to longer fixation times, low-frequency words often
induce immediate regressions back to earlier parts of the sentence; 90 per cent
of regressions are the result of comprehension and 10 per cent occur because of
corrective ocular movements (Staub and Rayner 2007).
Regressive eye movements are therefore of particular relevance to this
experiment, which aims to analyse the effect that the inclusion of txt forms
in subtitles has on how they are read. A significant increase in the number of
regressions in the subtitles containing textisms would indicate difficulty in reading
and decoding the subtitles in question.
Regression data is, however, complemented here by fixation count results
and mean fixation duration data, with a view to drawing conclusions on both
comprehension and cognitive effort.
Data analysis
For data analysis, we used the IBM SPSS Statistics 20 package and the Tobii
Studio statistics package. Following close analysis of the experimental data,
several statistical tests were selected. The significance level used in the present
experiment for all statistical tests is 0.05, which is the most widely used level of
significance (Lang 2007). Reported p-values fall into four categories ranging from
non-statistically significant results (p > 0.05) to statistically significant (p < 0.05,
p < 0.01 and p < 0.001).
Effect will be reported at a 0.05 level of significance as well and the r value
was calculated using the online UCCS tool.2 The guidelines for interpreting the
r value are 0.10 for small effect, 0.30 for medium effect and 0.50 for large effect
(Field and Hole 2012: 153).
Results of the statistical analysis for the regressions identified in the two
groups and information on fixation count and fixation duration follow. Two sets
of values are provided; one set allows the assessment of similarity in reading
ability and viewing behaviour of all participants under normal subtitling
conditions. To this effect, data points from 22 traditionally created subtitles with
standard spelling, viewed by all eight of our participants, were used and statistical
values were provided. This step is labelled the ‘Control’ condition and its only
function is to measure whether, in normal subtitling situations, all participants
display similar characteristics when it comes to regressions, fixation count and
fixation duration. The absence of significant differences between participants
under normal subtitling conditions would enable us to more confidently interpret
possible differences observed in these three parameters when txt lingo is included
Cn U read ths? 159
as variable in the ‘Experiment’ stage. If the values for the three parameters were
to differ significantly under normal conditions, a clear link between possible
observed differences and the variable would be difficult to make. Having thus
assessed all participants’ viewing patterns, for the second stage, the ‘Experiment’
condition, the participants were split into two groups, the non-treatment (C1)
and the treatment (C2), and data points from 136 subtitles were analysed. C1
participants watched subtitles created using normal, standard spelling, while C2
watched the same subtitles, but this time created using txt lingo.
Certain limitations in the research design need to be acknowledged. The present
study is limited in its capacity to make refined observations regarding the reading
process. It was beyond its scope to investigate reading behaviour by combining
different eye-tracking measures, for example processing speed measurements
with pupil dilation data (O’Brien 2008), or to use a complex model such as
the reading index for dynamic texts suggested by Kruger and Steyn (2013).
However, when putting forward ideas regarding ease of processing of txt lingo,
regression data will be linked to analysis of fixation count and mean fixation
durations, which are known measures of cognitive effort. Moreover, although
the number of participants was low, it was generally in line with previous similar
studies (Neville 2003; Caffrey 2009; O’Brien 2009; Perego et al. 2010). Given
that access to a large number of participants is usually beyond most researchers’
means, we believe this sample size to be suitable for the purpose of the study
and, as noted by O’Brien (2009: 255), while generalizations based on such small
numbers are questionable, these pioneering studies can generate ‘hypotheses
using small communities which can then be tested on larger communities’.
Results
When gathering quantitative data on regressions, we differentiated between
regressions proper and second-pass reading, which ‘includes all fixations after
a regressive eye movement on those parts of the text that the eye had already
passed during the first pass’ (Wotschack 2009: 5). For two-line subtitles, we
also excluded those regressive movements to the first line when these resulted
in an immediate shift – that is, we considered such movement to be linked to the
vertical axis and not the horizontal one. Nevertheless, when such a movement
did not result in an immediate exit from the subtitle AOI, we considered it to be
a regression and therefore included it in the count. For example, based on the
method described, the subtitle in Figure 8.3 includes two regressions: 7 to 8;
and 9 to 10. The path 8 to 9 is not a regression, even if the area had already been
fixated upon once at 7, but a second-pass reading that follows logically from 8.
Data for the participants reading subtitles under the Control normal condition
revealed that participants who were part of C1 in the Experiment performed a total of
67 regressions in the 22 subtitles shown, while C2 participants had a total of 70 – that
is, only a small difference of 4.285 per cent was recorded. The 88 data points included
in the statistical analysis for each group generated the results in Table 8.4 (in which ‘M’
refers to mean, ‘SD’, to standard deviation, and ‘ME’, to standard error of the mean).
160 Alina Secară
Figure 8.3 Regressions, group C2, participant 1
Table 8.4 Regressions t-test results in Control
Control M SD SE
C1 0.7614 0.90965 0.09697

C2 0.7955 0.96083 0.10242
Note: t (174) = –0.242; p = 0.809; r = –0.0182
This indicates that, under normal subtitling conditions, there is no statistically

significant difference between the two participant groups in terms of regressions.
In the Experiment data, on the other hand, the treatment group C2 was found to
have a total of 813 regressions compared to 583 regressions in the non-treatment
group C1. This signifies an increase of more than 28 per cent in the number of
regressions for the group exposed to txt subtitles. The Shapiro–Wilk value is
significant (p < 0.05) and a histogram, Figure 8.4, was produced to visualize the
distribution of the data.
In the Mann–Whitney U test, the treatment group (Mdn = 1.0) significantly
differs from the non-treatment group (Mdn = 1.0) regarding the regression count
(U = 121909.500 and z = –5.233, with a significance level of p = 0.000, r = –0.178,
therefore significant, with a small effect).
The descriptive statistics and the independent samples t-test also show a
significant difference between the two groups and a medium effect (Table 8.5).
Cn U read ths? 161
Figure 8.4 Histogram for regressions in the two Experiment groups
Because there was no difference recorded between the two groups in the
Control stage, the statistically significant result recorded in the Experiment
condition would appear to be linked to the presence of txt language.
The number of skipped subtitles represents another indication of a lack of
coherent distribution of gaze while reading subtitles. There were very few instances
of skipped subtitles; for both groups, the values recorded were identical – namely,
three skipped subtitles in the first part of the file and only one in the second. There
are substantial differences between the two groups when it comes to another metric,
however: the number of instances in which participants failed to watch outside
of the subtitling area (AOI). In the treatment condition group, across the four
participants, there were 151 instances (69 in the first part of the file and 82 in the
second) in which participants did not have time to focus their attention on the screen
anywhere outside the subtitling area. For the non-treatment condition, the figure
was significantly lower, at 67 (27 in the first part of the file and 40 in the second). It
is therefore apparent that, in the treatment condition, viewers’ visual resources were
allocated entirely to the processing of the linguistic information found in subtitles in
more instances than was the case in the non-treatment condition.
This is further supported by results for fixation counts and mean fixation
durations. The independent-samples t-test shows no significant difference
between the participants under normal Control subtitling conditions (Table 8.6).
162 Alina Secară
Table 8.5 Regressions t-test results in Experiment
Experiment M SD SE
C1 1.0717 0.95208 0.04082

C2 1.4945 1.25701 0.05389
Note: t (1086) = 6.254; p = 0.000; r = –0.186
Table 8.6 Fixation count t-test results in Control
Control M SD
Inside AOIs
C1 6.8068 4.14357
C2 6.7500 4.06343
Outside AOIs
C1 2.7386 1.70526
C2 2.2614 1.48164
Notes
Inside AOIs: t (174) = 0.092; p = 0.927; r = 0.0069
Outside AOIs: t (174) = 1.982; p = 0.051; r = 0.1477
Table 8.7 Fixation count t-test results in Experiment
Experiment M SD SE
Inside AOIs
C 9.4559 4.10294 0.17591

C2 9.1783 3.98656 0.17092
Outside AOIs
C1 2.7077 1.96213 0.08413

C2 2.2420 1.62093 0.06950
Notes
Inside AOIs: t (1086) = 1.132; p = 0.258; r = 0.034
Outside AOIs: t (1086) = 4.268; p = 0.000022; r = 0.0128
When the Experiment data is analysed (Table 8.7), the t-test shows no significant
difference between the two groups inside the subtitling area, but signals statistical
difference outside the subtitling area. Fewer fixations were recorded for participants
in C2 outside of the subtitling area – a fact that indicates a less thorough processing
of the information appearing in areas in which no subtitles were found.
The values for the mean fixation duration in Experiment inside the AOIs were
M = 0.2115 for C1 and M = 0.2607 for C2 (t (1078) = –8.675, p = 0.001, r = –0.255).
The mean fixation duration value for the area outside of the AOIs is also higher for
Cn U read ths? 163
C2 at M = 0.3153, but with fewer fixations, versus C1 M = 0.2437 (t (874) = –2.895,
p = 0.004, r = –0.094. The difference between C1 and C2 participants for this variable
is therefore statistically significant for both inside and outside the AOI categories. The
C2 mean fixation duration inside the AOIs is indeed higher than the mean fixation
duration for C1, which would indicate a more intense reading experience.
In summary, the results presented here suggest that the introduction of the
txt lingo variable in the subtitles led to an increase in viewers’ cognitive effort,
as measured by the three parameters: regressions, fixation count (in particular
fixation count outside the subtitling AOI) and mean fixation duration.
Facilitation effect
Given the results from the regression data, which seem to indicate a certain
difficulty encountered by the C2 participants when watching a video containing
non-traditional spellings in its subtitles, we also wanted to explore the extent
to which exposure to subtitles containing txt lingo would have an effect on the
viewers’ reading behaviour. Specifically, we were interested in pursuing the
facilitation effect hypothesis, according to which exposure to a given stimulus
leads to ease in its processing (Kahneman 2012).
We conducted both inter-group and intra-group analyses of our regressions
data. For the inter-group analysis, data was analysed for the last 68 subtitles in
each condition file. The intra-group analysis consisted of an investigation of the
gaze data from the first part of the subtitling file for C2 in comparison with values
taken from the second part of the file, for the same group. This allowed us to draw
conclusions regarding the changes that took part within the treatment group over
the course of the experiment.
In the inter-group analysis, there were significant differences recorded for
regressions (M = 1.0368, SD = 0.89614, SE = 0.05434 for group C1; M = 1.6250,
SD = 1.40604, SE = 0.08525 for C2; t (542) = –5.819, p = 0.000, r = –0.242). Thus,
when only the last part of the subtitling file is analysed, we see that regression
values remain statistically different between the two groups, as in the Experiment
analysis considered in the last section.
For the intra-group analysis, the values increased from M = 1.3640 in the first
part of the file to M = 1.6250 in the second, highlighting a much more intense
visual activity on the horizontal line within AOIs in the second part of the file.
The increase in the regression activity in the second part of the file is represented
in two gaze plots from the same participant in Figures 8.5 and 8.6.
For mean fixation durations in the inter-group analysis, no significant
difference was recorded inside or outside the AOI.3 For the intra-group analysis
(within C2), however, a facilitation effect can be inferred from the figures for
mean duration of fixations inside the AOIs, which registered a significant fall
(from M = 0.2780 to M = 0.2323).
Likewise, no significant difference was recorded for fixation count in the inter-
group analysis.4 For the intra-group analysis, a slight increase was recorded for
fixation count, from M = 9.0368 to M = 9.3199 inside the AOI, and from M =
2.2047 to M = 2.2794 outside the subtitling area.
Figure 8.5 Gaze path of C2 participant, first part of the file
Figure 8.6 Gaze path of C2 participant, second part of the file

Cn U read ths? 165
These results are interpreted here as evidence of an improved viewing experience
for C2 viewers: not only did the difference in mean fixation durations between
the two groups disappear in the second part of the file, but also C2 participants
increased their fixation counts outside the subtitling area in the second part of the
file. It appears that not only were viewers getting more comfortable reading in
the subtitling environment overall and feeling more comfortable distributing their
attention more evenly on the screen, but also that they were getting more familiar
with, and consequently more proficient at consuming, txt lingo.
Comprehension and attitude questionnaire

A multiple-choice questionnaire was used immediately after the experiment to
test participants’ comprehension of the subtitles and attitudes to txt language in
subtitling. Each question offered three possible answers, with only one being correct.
For data analysis, each correct answer was given a one-point score; each incorrect
answer, zero points. Ten questions tested overall comprehension of the clip. SPSS
output for the non-parametric Mann–Whitney U test shows a non-significant two-
tailed probability (0.579 > 0.05), which would indicate that the answers provided
by the two groups were close. This is also supported by the descriptive statistic test,
with values of M = 2.8000, SE = 0.3887 for the treatment group and M = 3.2000,
SE = 0.2494 for the non-treatment group, reflected in participants’ self-evaluation
of their comprehension and reading ease. None of the eight participants reported
problems regarding comprehension, with five responding that they understood
most subtitles and three all subtitles. Likewise, all participants reported that they
Could read most of the subtitle before it went off screen. Based on this data, the
presence of txt lingo does not seem to have had a big impact on the participants’
self-reported understanding or reading of the subtitles.
Despite the positive self-reported txt language proficiency and the
questionnaire-based comprehension results, the answers to questions regarding
preference in terms of presentation of subtitles (with or without txt lingo) and
associated status of txt lingo generally showed very low tolerance of non-standard
practices. All participants indicated that they would prefer to watch a foreign-
language clip on the Internet with standard subtitles. Moreover, when asked
how they rate information written in txt language in general, all participants
answered of inferior quality to content written in standard form. Overall, there
was no difference between the two groups regarding tolerance of txt lingo and
comprehension ability or previous exposure to txt lingo seemed to have no effect
on the participants’ view of the status of this practice.
Conclusions
By introducing txt lingo practices in a subtitling context, we seem to have initially
disrupted our viewers’ reading experience, despite the fact that the viewers were
frequent producers and consumers of textese (as established in our screening
phase). It is possible that these practices are so ingrained in the contexts in which
they normally occur that their displacement to a new context automatically
166 Alina Secară
triggers processing difficulties and rejection. Barton and Lee (2013) insist on the
situated nature of language, and taking a practice outside its normal boundaries
may trigger a lack of control of such practices (Wenger 1998). This may explain
the overwhelming rejection of txt practices in the questionnaire. Although all
participants included in the treatment group were frequent txt consumers and
producers, their reaction to the possibility of consuming more txt language in
subtitling, and across a greater variety of contexts, was very conservative and
similar to that of the C1 participants. All participants considered information
written in txt lingo to be Of inferior quality to content written in standard form;
they all expected to find it, first and foremost, on the Internet and in personal
communication contexts. Their answers did not logically follow the results of the
comprehension questionnaire data, which suggested no difference between the
two groups. We therefore believe that the suggestion of taking this practice from
a specific community, such as IM or social translation platforms, and transferring
it to a more open medium could have been interpreted as threatening. That said,
viewers adapt to visual stimuli quickly and, given that all participants were
frequent consumers of subtitles, we can imagine that further exposure to txt lingo
in subtitling contexts would lead to increased acceptance.
Regarding the actual readability of textisms in our experiment, one possible
explanation for the difficulty indicated by regressions and longer mean fixation
durations is that the reading of the txt forms may require a number of strategies
that are demanding of resources. Reinstating vowels where they are missing, for
example in forms such as ez and dnt, or substituting in homophonic morphemes
for letters or numbers such asd 4 or c, are examples of such strategies. Switching
between writing styles has been proven to incur higher processing demands
(see Kemp 2010). The difference between time spent on producing and reading
textisms and time spent learning and using conventional forms is very large, even
for frequent texters, but it is possible, as suggested by some pioneering studies
(Plester et al. 2009), that these differences may begin to fall if exposure to both
begins at an early age. The encouraging results obtained in our study for inter- and
intra-group facilitation effects also seem to support this view.
For the moment, slight disadvantages in processing may be seen as a trade-off
for the sense of social inclusion, intimacy and fun that users of these linguistic
forms have been reported to enjoy (Baron 2010a, 2010b). Moreover, there might
be further benefits from this initial discomfort. Recent work into disfluency, the
‘subjective experience of difficulty associated with cognitive operations’, suggests
that additional cognitive burdens may lead to improved learning (Diemand-Yauman
et al. 2010: 111). In a pioneering work on the impact of manipulation of hard-to
read fonts, Diemand-Yauman and colleagues (2010: 112) stress that it is not the
difficulty of the task in itself that leads to benefits in learning, but the processes that
this difficulty engenders, with participants ‘less likely to use heuristics’ and relying
‘on more systematic and elaborative reasoning strategies’. This may explain the
similarity between our two groups in the comprehension questionnaire results.
The findings of this study highlighting positive changes in mean fixation
duration and fixation counts outside of the subtitling areas after a certain amount
Cn U read ths? 167
of exposure to stimuli containing subtitles in txt lingo, as well as our previous
work (Secară 2011), along with examples highlighted in this chapter regarding
the positive correlation between the use of textisms and literacy, spelling
proficiency and reading scores, are all encouraging of discussion and, hopefully,
future research into the potential of txt lingo. While txt lingo may still have low
status, even among our participants, we have seen how practices such as the use
of emoticons have become more accepted; the same could happen with other
txt lingo practices. Increased production and exposure to txt language practices
could lead to a less judgemental approach and a readier acceptance of this
currently non-conformist practice. Despite the facilitation effect observed in our
study, however, all participants still spent more than 60 per cent of the subtitle
display time in the subtitling area, with higher values recorded for C2 than C1
participants. Given that the subtitling area covers less than a third of the screen, the
disproportionate, yet necessary, allocation of visual resources to this area should
be more seriously considered, especially in the context of the viewers’ overall
enjoyment of subtitled films. With the advent of hybrid broadcast broadband
television (HbbTV) (Tai 2014), which allows for true subtitle customization, the
standard approach to subtitling should now be more vocally challenged. A recent
study (Fox 2014) suggests the advantages that such customization can have on
the reading experience for subtitling viewing and is consistent with the central
idea of our study – namely, that professional practices need to be challenged, and
must respond to changes that are taking place in the field and in society in general.
Notes
1 Transl8it! is a free online platform that converts plain English into txt lingo and vice
versa. The platform allows users to contribute to the database of available txt forms and
also to correct txt forms suggested.
2 See http://www.uccs.edu/~lbecker/ [accessed 12 March 2016].
3 The values for mean fixation duration inside the AOI for the C1 groups were M = 0.2161,
SD = 0.13493, SE = 0.00820, and for the C2 group, M = 0.2323, SD = 0.07240, SE = 0.00440
(t (542) = –1.738, p = 0.083, r = –0.074). For the mean fixation duration outside the AOI for
the C1 group, the analysis shows M = 0.2468, SD = 0.15371, SE = 0.01009 and M = 0.2706,
SD = 0.18664, SE = 0.01228 for the treatment group; hence not significant (t (542) = –1.500,
p = 0.134 and r = –0.0694).
4 The fixation count inside the AOI had values of M = 9.3456, SD = 4.02827, SE = 0.24425
(for C1) versus M = 9.3199, SD = 3.95042, SE = 0.23953 (C2) (t (542) = 0.075, p = 0.940,
r = 0.00322). For outside the AOI, the fixation count values were M = 2.5147, SD = 1.90158,
SE = 0.11530 (C1) and M = 2.2794, SD = 1.58959, SE = 0.09638 (C2) (t (542) = 1.566,
p = 0.118, r = 0.0669).
References
Baron, S. N. 2010a. ‘Discourse Structures in Instant Messaging: The Case of Utterance
Breaks’. Language@Internet 7. Available at: http://www.languageatinternet.org/
articles/2010/2651 [accessed 12 March 2016].
Baron, S. N. 2010b. Always On: Language in an Online and Mobile World. Oxford:
Oxford University Press.
168 Alina Secară
Barton, D., and C. Lee. 2013. Language Online: Investigating Digital Texts and Practices.
London/New York: Routledge.
Caffrey, C. 2009. Relevant Abuse? Investigating the Effects of an Abusive Subtitling
Procedure on the Perception of TV Anime Using Eye Tracker and Questionnaire. PhD
thesis. Dublin City University. Available at: http://doras.dcu.ie/14835/ [accessed 12
March 2016].
Carrington, V. 2004. ‘Texts and Literacies of the Shi Jinrui’. British Journal of Sociology
of Education 25(2): 215–28.
Civera, C. 2005. ‘Introducing Emoticons and Pictograms in SDHH’. Paper presented at the
Media for All Conference, Barcelona, 6–8 June.
Cobb, T. 2008. Web Vocabprofile. Available at: http://www.lextutor.ca/vp/ [accessed 12
March 2016].
Coventry University. 2009. ‘Coventry University Finds that Texting Actually
Improves Language Skills’. News release. Available at: http://www.coventry.ac.uk/
latestnewsandevents/a/2341 [accessed 12 March 2016].
Crystal, D. 2006. Language and the Internet. Cambridge: Cambridge University Press.
Crystal, D. 2009. The Future of Language. London/New York: Routledge.
Diemand-Yauman, C., D. M. Oppenheimer and E. B. Vaughan. 2010. ‘Fortune Favors the
Bold (and the Italicized): Effects of Disfluency on Educational Outcomes’. Cognition
118(1): 111–15.
Ferraresi, A., E. Zanchetta, M. Baroni and S. Bernardini. 2008. ‘Introducing and Evaluating
ukWaC, a Very Large Web-Derived Corpus of English’. Paper presented at the
sixth international conference on Language Resources and Evaluation (LREC2008).
Marrakech, 26 May–1 June. Available at: http://clic.cimec.unitn.it/marco/publications/
lrec2008/lrec08-ukwac.pdf [accessed 12 March 2016].
Field, A., and G. Hole. 2012. How to Design and Report Experiments. London/New Delhi:
Sage.
Fox, W. 2014. ‘Film Identity, Reception, Aesthetics and Information Flow: Integrated
Titles and Other Possible Improvements’. Paper presented at the Languages and the
Media Conference 2014, Berlin, 5–7 November.
Fullwood, C., L. Orchard and S. A. Floyd. 2013. ‘Emoticon Convergence in Internet Chat
Rooms’. Social Semiotics 23(5): 648–62.
Ghia, E. 2012. Subtitling Matters: New Perspectives on Subtitling and Foreign Language
Learning. Bern: Peter Lang.
Goldstein, B. R., R. L. Woods and E. Peli. 2007. ‘Where People Look when Watching
Movies: Do All Viewers Look at the Same Place?’ Computers in Biology and Medicine
37(7): 957–64.
Hefer, E. 2010. Reading Second-Language Subtitles: A Case Study of Afrikaans Viewers
Reading in Afrikaans and English. MA thesis. Vaal Triangle Campus of the North-
West University. Available at: https://dspace.nwu.ac.za/handle/10394/12138 [accessed
12 March 2016].
Howell, P. 2006. ‘Character Voice in Anime Subtitles’. Perspectives: Studies in
Translatology 14(4): 292–305.
Humphrys, J. 2007. ‘I h8 Txt Msgs: How Texting is Wrecking Our Language’. Daily Mail,
24 September. Available at: http://www.dailymail.co.uk/news/article-483511/I-h8-txt-
msgs-How-texting-wrecking-language.html [accessed 12 March 2016].
Jacquemin, L., and D. Pinson. 2009. Surfeurs du paradis. Papeete/Nouméa: Archipel.
Kahneman, D. 2012. Thinking, Fast and Slow. London: Penguin.
Cn U read ths? 169
Kemp, N. 2010. ‘Texting versus Txtng: Reading and Writing Text Messages, and Links
with Other Linguistic Skills’. Writing Systems Research 2(1): 53–71.
Kruger, J.-L., and F. Steyn. 2013. ‘Subtitles and Eye Tracking: Reading and Performance’.
Reading Research Quarterly 49(1): 105–20.
Lang, T. 2007. ‘Documenting Research in Scientific Articles: Guidelines for Authors, 2 –
Reporting Hypothesis Tests’. Chest 131(1): 317–19.
Ling, R., and N. S. Baron. 2007. ‘Text Messaging and IM: A Linguistic Comparison of
American College Data’. Journal of Language and Social Psychology 26(3): 291–8.
Neville, L. 2003. Cn U rEd dis? The Causes and Consequences of a ‘Text Message
Language’ in Young Teenagers. Undergraduate dissertation. University of Oxford.
Nornes, A. M. 1999. ‘For an Abusive Subtitling’. Film Quarterly 52(3): 17–34.
O’Brien, S. 2008. ‘Processing Fuzzy Matches in Translation Memory Tools: An Eye-
Tracking Analysis’. In S. Göpferich, A. L. Jakobsen and I. M. Mees (eds). Looking
at Eyes: Eye Tracking Studies of Reading and Translation Processing (Copenhagen
Studies in Language 36). Copenhagen: Samfundslitteratur, 79–102.
O’Brien, S. 2009. ‘Eye Tracking in Translation-Process Research: Methodological
Challenges and Solutions’. In I. M. Mees, F. Alves and S. Göpferich (eds). Methodology,
Technology and Innovation in Translation Process Research (Copenhagen Studies in
Language 38). Copenhagen: Samfundslitteratur, 251–66.
O’Hagan, M. 2008. ‘Fan Translation Networks: An Accidental Translator Training
Environment?’ In J. Kearns (ed.) Translator and Interpreter Training: Issues, Methods
and Debates. London/New York: Continuum, 159–83.
Ofcom. 2013. ‘The Communications Market Report: Scotland’. Available at: http://
stakeholders.ofcom.org.uk/market-data-research/market-data/communications-
market-reports/cmr13/scotland/ [accessed 12 March 2016].
Perego, E., F. Del Missier, M. Porta and M. Mosconi. 2010. ‘The Cognitive Effectiveness
of Subtitle Processing’. Media Psychology 13(3): 243–72.
Pérez González, L. 2006. ‘Fansubbing Anime: Insights into the “Butterfly Effect” of
Globalisation on Audiovisual Translation’. Perspectives 14(4): 260–77.
Plester, B., C. Wood and P. Joshi. 2009. ‘Exploring the Relationship between Children’s
Knowledge of Text Message Abbreviations and School Literacy Outcomes’. British
Journal of Developmental Psychology 27(1): 145–61.
Pollatsek, A., and K. Rayner. 2006. ‘Eye-Movement Control in Reading’. In M. J. Traxler
and M. A. Gernsbacher (eds). Handbook of Psycholinguistics. 2nd edn. San Diego, CA:
Elsevier, 613–57.
Rayner, K., and A. Pollatsek. 1989. The Psychology of Reading. Hillsdale, NJ: Lawrence
Erlbaum Associates.
Rayner, K., S. P. Liversedge and S. J. White. 2006. ‘Eye Movements when Reading
Disappearing Text: The Importance of the Word to the Right of Fixation’. Vision
Research 46(3): 310–23.
Secară, A. 2011. ‘R U Ready 4 New Subtitles? Investigating the Potential of Social
Translation Practices’. Linguistica Antverpiensia 10: 153–74.
Shortis, T. 2007. ‘Revoicing Txt: Spelling, Vernacular Orthography and “Unregimented
Writing” ’. In S. Pasteguillo, M. J. Esteve and M. L. Geo-Valor (eds). The Texture of
Internet: Netlinguistics in Progress. Newcastle: Cambridge Scholars, 2–20.
Staub, A., and K. Rayner. 2007. ‘Eye Movements and Online Comprehension Processes’.
In G. M. Gaskell (ed.). The Oxford Handbook of Psycholinguistics. Oxford: Oxford
University Press, 327–42.
170 Alina Secară
Tagg, C. 2009. A Corpus Linguistics Study of SMS Text Messaging. PhD thesis.
Department of English, University of Birmingham. Available at: http://etheses.bham.
ac.uk/253/ [accessed 12 March 2016].
Tagliamonte, S. A., and D. Denis. 2008. ‘Linguistic Ruin? Lol! Instant Messaging and
Teen Language’. American Speech 83(1): 3–14.
Tai, A. 2014. ‘Opportunities and Challenges of Web Captions for Broadcast Video
Content’. Paper presented at the Languages and the Media Conference 2014, Berlin,
5–7 November,
Wenger, E. 1998. Communities of Practice: Learning, Meaning, and Identity. Cambridge:
Cambridge University Press.
Werry, C. C. 1996. ‘Linguistic and Interactional Features of Internet Relay Chat’. In S.
Herring (ed.). Computer-Mediated Communication: Linguistic, Social and Cross-
Cultural Perspectives. Amsterdam: John Benjamins, 47–64.
Wotschack, C. 2009. Eye Movements in Reading Strategies: How Reading Strategies
Modulate Effects of Distributed Processing and Oculomotor Control. Potsdam:
Universitätsverlag.
Zarate, S., and J. Eliahoo. 2014. ‘Word Recognition and Content Comprehension of
Subtitles for Television by Deaf Children’. Journal of Specialised Translation 21:
133–52.
Index
acceptability 131, 143n1 Baidu dictionary 63, 68, 69, 70, 71, 73
accuracy 131, 134 Baron, S. N. 151, 154
acronyms 154 Barton, D. 166
actor-network theory (ANT) 4, 26, 31–6, Bergenholtz, H. 80, 81
37, 39–40 bias 141–2
adequacy 131, 133 Biau Gil, J. R. 80
administrative practices 3, 45, 57 ‘big data’ 136
advertising 151 bilingual evaluation understudy (BLEU)
AEMs see automation evaluation metrics 134–5
agency 26, 31, 38, 40; actor-network Brabham, D. 29
theory 32; ‘dance of agency’ 64–5, 74, ‘break-up letters’ 4, 9, 10–23
76 Brunello, M. 135–6
Ai Ciba 63 business and administrative practices 3,
Alabau, V. 121 45, 57
Alchemy Catalyst 116, 123 Buzelin, H. 32, 40
Alchemy Publisher 116 Byrne, J. 80, 86
algorithms 1, 38, 39 Bystrova-McIntyre, T. 133
Amazon’s Mechanical Turk (AMT) 29,
30, 136 Caitra 112
American Translators Association (ATA) 27 Callison-Burch, C. 30
Anckaert, P. 133 Callon, M. 32
anime 153 Canada 4, 8, 15, 45, 46, 58–60, 63
ANT see actor-network theory Casmacat 112, 126
Appraise 136 CAT see computer-assisted translation
ATA see American Translators CERTT see Collection of Electronic
Association Resources in Translation Technologies
attitudes to technology 10, 13–23 Chinese 5, 63–4, 65–76
Australia 66, 150 cloud-based services 27, 135
automation 56, 58, 132 CMC see computer-mediated
automation evaluation metrics (AEMs) communication
134–5, 137, 141 cognitive effort 143n1, 143n4, 158, 159,
autonomy 4, 26, 31, 40, 45; crowdsourcing 163, 166
36–8, 39; loss of 55–6; lower autonomy cognitive functions 81–2, 84, 98, 104
perception 27–8, 60 collaborative translation 26
Collection of Electronic Resources in
Bachman, L. 138, 139 Translation Technologies (CERTT)
Baer, B. 133 18–19
172 Index
collective intelligence 29, 30 Denis, D. 154
Common Sense Advisory 141 DePalma, D. 137
communicative functions 81–2, 98, 104 deprofessionalization 55–6, 60
community translation 26 Désilets, A. 63
comprehension 157–8, 165 deskilling 28, 48
computer-assisted translation (CAT) DGT see Directorate-General for
40, 45, 65, 80; Facebook translation Translation
crowdsourcing 39; Matecat project 112; dialects 152
post-editing user interfaces 111, 117, dictionaries 3, 45, 80, 82; division of
119; technologization of translation knowledge in 92–4; English-Chinese
27; translation quality assessment 137; translation web resources 5, 63, 67–70,
trust 29; see also machine translation; 71, 73, 74; functions 81, 83; ‘letters’ to
software; translation memory 14; lexicographic needs and preferences
computer-mediated communication 83–5, 89, 90–7; user interfaces 117, 118
(CMC) 150 Diemand-Yauman, C. 166
concept maps 86, 88, 95–6, 98 Dillon, Sarah 8, 17
conceptual networks 86, 87, 95–6, 98 Directorate-General for Translation (DGT)
confidence scores 29, 120, 125, 127n2 15, 22
consistency 47, 48 discussion forums 92, 94
constructivism 2, 3 disfluency 166
context: eye-tracking data 158; division of labour 58
lexicographic tools 83; txt language 151, Doherty, Stephen 5–6, 131–48
165–6 Dombek, M. 33, 37, 41n5, 41n6
control, loss of 59; see also autonomy Drugan, J. 38, 39
corpora 135–6, 154, 155 Durán Muñoz, I. 83
creativity 31, 48
credibility 36, 38 ease of use 9–10, 23; see also usability
Cresswell, K. M. 40 EcoLexicon 82–3, 86, 96, 103–4
critical theory of technology (CTT) 2 economic control 3
Cronin, M. 2 efficiency 9–10, 19–22
crowdsourcing 4, 25–6, 28; cloud effortlessness 9–10
marketplaces 27; evaluation 136, 142; Ehrensberger-Dow, M. 126
Facebook 25–6, 32–9; research on email 151
29–31; subtitling 149; trust 29 emoticons 152, 167
Crystal, D. 151 emotions 4, 8–23; research methodology
CTT see critical theory of technology 10–13; technology acceptance 9–10;
time, experience and technological
Damasio, A. 9 change 16–19; usability and user
‘dance of agency’ 64–5, 74, 76 experience 19–22
data analysis 67, 158–9 encyclopaedias 83–5, 89, 90–2, 93
data collection: eye-tracking data 6, 153, English-Chinese translation 5, 63–4, 65–76
156–8; post-editing user interfaces errors 9, 19–22; Facebook translation
112–14; translators’ emotions 11; see crowdsourcing 37; post-editing 109,
also interviews; research methods; 117, 121, 124; recycling of existing
surveys translations 54; translation memory
data dispossession 122 systems 48; translation quality
databases 14, 15, 19, 48 assessment 132, 137; Workshop in
De Almeida, G. 111 Machine Translation 136
democratization 3, 36–7 Estellés-Arolas, E. 30
Index 173
ethics 27, 31, 46, 157 Google Translate 15, 68, 71, 73, 124–5
ethnography 47 Gouadec, D. 58, 60
EU Directorate-General for Translation grammar 124
(DGT) 15, 22 Grave of the Fireflies 153
European Union (EU) 11, 14, 15, 16, 17, Green, S. 139
23n4 Grimes, S. M. 2
evaluation 5–6, 131–48; evaluators 140–2; GTM see general text matcher
in industry 136–7; machine translation guidelines 48–9
research 133–6; materials 142–3;
psychometrics 138; reliability 139–40; Haici dictionary 68, 69, 73
translation studies 132–3; validity 138–9 hardware 13, 14
experience of translators 17–18, 61n5; HCI see human-computer interaction
lexicographic needs and preferences 90; Hefer, E. 157–8
post-editing 114, 123; web resources 66, ‘hi-tech lingo’ 151
72; see also junior translators; senior HITs see human intelligent tasks
translators Hossain, M. 29–30
Eyckmans, J. 133 House, J. 133
eye-tracking data 6, 153, 156–8, 159–65, Howe, Jeff 25, 29
166–7 Howell, P. 153
Hubscher-Davidson, S. 47
Facebook 2, 4, 25–6, 27, 32–9, 41n5, 41n7 human-computer interaction (HCI) 64–5,
facilitation effect 163–5, 166, 167 126–7
fan translation 25, 149, 151, 152–3 ‘human intelligence’ 29, 30, 31
FBT see frame-based terminology human intelligent tasks (HITs) 29, 30
Feenberg, A. 2
Finland 11, 14, 15, 16, 23n4 ichacha 68
Flesch reading ease index 155 iconicity 152
fluency 131, 133, 134 Idiom Worldserver 118, 123
Flynn, P. 47 illustrations 94–5
frame-based terminology (FBT) 5, 81, IM see instant messaging
82–3, 97, 104 images 94–5
Fraser, Janet 8, 17 IMT see interactive machine translation
function theory of lexicography (FTL) 5, industrialization 56, 58
81–2, 83, 97, 104 information costs 80
functionalist approaches 81, 82, 83, 103 information, offers of 82
fuzzy matches 49, 50, 52, 53, 110, 120 infoxication 80
innovation: distributed 29; open 30;
García-Aragón, Alejandro 3, 5, 80–108 ‘pro-innovation bias’ 1
Garcia, I. 2, 28 instant messaging (IM) 6, 150, 151, 152,
Gaspari, F. 137 154, 166
gender 124, 125 intellectual property 121
general text matcher (GTM) 134, 135 inter-rater reliability 139–40, 142
generational differences 16–18 interactive machine translation (IMT) 111,
Gengo 27 121–2
Ghia, E. 157 interlingual subtitles 6, 149–50, 152–67
glossaries 117, 125 Internet 22, 25, 29; actor-network theory
Goldstein, B. R. 149 32; English-Chinese translation web
González-Ladrón-de-Guevara, F. 30 resources 5, 63–4, 65–76; ‘letters’ to
Google 20–1, 69, 71, 73 the 14, 18, 19; subtitling 149–50; txt
174 Index
language 152; see also crowdsourcing; translators’ needs and preferences 5,
search tools 80–1, 83–97, 100–1, 104
interviews 4, 47, 66–7, 113–14, 122–5, 126 Lilt 126
iOmegaT 111–12 Lim, Lily 2, 3, 5, 63–79
Ireland 11 LimeService 113
LimeSurvey 85
Japanese anime 153 Lionbridge Enterprise Crowdsourcing 27
Jiu Hai 63 Lommel, A. 136, 141
JuKuu 63 López-Rodríguez, Clara Inés 3, 5, 80–108
junior translators: English-Chinese ‘love letters’ 4, 9, 10–23
translation web resources 5, 65–6, lower autonomy perception (LAP) 27–8, 60
67–72, 73, 74–6; impact of translation LSPs see language service providers
memory systems 51, 52, 55, 61n5; ‘love
letters’ 16–18; see also students machine translation (MT) 3, 8;
English-Chinese translation web
Karamanis, N. 28 resources 5; interactive 111, 121–2;
Katan, D. 27–8, 38 online 25, 35; post-editing user
Kauranen, I. 29–30 interfaces 109–27; retraining 121, 126;
Keen, A. 27 technologization of translation 27;
Kelly, N. 109, 137 translation quality assessment 131,
Kemp, N. 155 133–6, 137, 143n1; translators’ letters
Kenny, Dorothy 1–7, 135–6 15; trust 29
keyboard shortcuts 118–19 Machine Translation (journal) 140–1
knowledge: division according to ‘mangle of practice’ concept 64–5, 74, 76
user expertise 92–4; MeteoTrad 98; Marshman, E. 59
specialized 96, 97 Masashi, Yamamoto 153
Koch, G. 140 Massey, G. 126
Koponen, M. 111 Matecat 112, 126
Koskinen, Kaisa 3, 4, 8–24 McBride, C. 111
Kruger, J.-L. 159 meaning 82, 133–4
MemoQ 15, 23n5, 118, 122, 123, 124
Labaka, G. 135 memorability 9, 19–22
Lagoudaki, E. 23n5, 111, 126, 127n5, 127n6 memory 72
Landis, J. R. 140 MeteoTrad 5, 81, 84, 89, 91, 96, 97–104
language service providers (LSPs) 26, micro-tasks 30, 31, 33
27, 31, 35, 38, 137; see also translation Microsoft Word 116, 118, 119
service providers Moorkens, Joss 2, 3, 5, 109–30
LAP see lower autonomy perception Morozov, E. 1–2
Latour, Bruno 2, 3, 4, 32 Mossop, B. 58
learnability 9, 19–22 MT see machine translation
LeBlanc, Matthieu 3, 4–5, 8, 12, 15, multimodality 152
45–62, 63
Lee, C. 166 Nakatsu, R. T. 30
León Araúz, P. 82 National Accreditation Authority for
letters 4, 9, 10–23 Translators and Interpreters (NAATI) 66
Levy, J. 104 natural-language processing (NLP) 135
lexicographic information costs 80 networks 25, 26; actor-network theory
lexicography: function theory of 5, 81–2, 32–3; conceptual 86, 87, 95–6, 98;
83, 97, 104; MeteoTrad 97–104; Facebook 34, 35
Index 175
Nielsen, Jakob 9, 19–20, 104 readability 143n1, 155
NLP see natural-language processing recall 134
non-professional translation 26 reception 84
non-verbal modalities 152 recycling 28, 45, 50–1, 53–5
Nornes, A. M. 153 reference works 3, 80; see also
dictionaries; encyclopaedias
objectivity 131–2, 135 regressive eye movements 157–8, 159–63,
O’Brien, Sharon 3, 5, 109–30, 159 166
offers of information 82 Reiss, K. 132–3
O’Hagan, Minako 2, 3, 4, 25–44, 153 reliability 132, 133, 137–8, 139–40, 141,
Olohan, M. 2, 64–5 142
OmegaT 112, 116, 122, 123 research methods 3–4; English-Chinese
open-source software (OSS) 30, 111–12 translation web resources 66–7; impact
orthography 6, 151 of translation memory systems 47;
Ozcelik, D. 111 ‘love letters’ 10–13; post-editing
user interfaces 112–14; survey on
Palmer, A. 138, 139 lexicographic needs and preferences
paradigmatic text processing 65, 71–2, 85–9; txt language and subtitling 150,
75–6 153–8
Passolo 118 researchers 141–2
PET 112 resistance 74
Pickering, Andrew 2, 64–5, 73, 74, 76 retraining 121, 126
Poland 33, 41n4 Risku, H. 40
Pollatsek, A. 158 Ruokonen, Minna 3, 4, 8–24
portals 63, 67–8, 74
post-editing 5, 109–27, 135; definition of Sánchez, José Manuel Romero 89
110; interview results 122–5; research satisfaction 19–22, 59, 60; MeteoTrad 103;
design 112–14; survey results 114–22; technology acceptance 9–10; translation
translation quality assessment 131, 139 memory systems 45, 48, 56, 57
precision 134 Saussure, F. de 65
‘pro-innovation bias’ 1 science and technology studies (STS) 1–3, 6
productivity 10, 22, 61n3, 82; impact of SDL Trados 15, 23n5, 65, 116, 118,
translation memory systems 45, 47, 48, 122–4, 127n5
49–50, 51–3, 56, 58; post-editing 118, SDLX 23n5, 118, 122, 123
122; translation quality assessment 131 SDT see self-determination theory
‘proliferation’ 50, 51 search tools: English-Chinese translation
provenance of data 117, 121, 126 web resources 66, 68–9, 71–2, 73;
psychometrics 138 ‘letters’ to 14, 15, 20–1; search engines 3
Pym, A. 28, 64, 65, 73, 76, 80 Secară, Alina 6, 149–70
segmentation 28, 48, 142
qualifications 66 self-determination theory (SDT) 37
quality assessment see translation quality senior translators: English-Chinese
assessment translation web resources 5, 66, 67–9,
quality assurance 38–9, 117 71, 72–3, 74–6; impact of translation
Quest 15, 22 memory systems 51, 52, 55, 56, 61n5;
questionnaire design 85–6 ‘love letters’ 16–18, 19
Short Message Service (SMS) texts 152,
Ray, R. 137 155; see also txt language
Rayner, K. 158 Shortis, T. 151
176 Index
simplicity 111, 122–3 translation 25, 26, 27–9, 31; translators’
Sleeswijk Visser, F. 127 emotions and attitudes to 4, 8–23; see
Smartling 27 also Internet; machine translation;
SMS see Short Message Service texts software; translation memory
social media 30 technoneutrality 1–2
software: involvement of end-users in technostructuralism 2
development 111; ‘letters’ to 13, 14, 15; Tehranian, M. 1–2
localization 37; open-source 30, 111–12; TER see translation edit rate
post-editing 116–17, 118, 123, 126; see term banks 45
also machine translation; search tools; termino-lexicographic functions 81–3, 84
technology; translation memory terminology 103; frame-based 5, 81, 82–3,
Somers, H. 110 97, 104; MeteoTrad 104; post-editing
space economy 152 user interfaces 125
specialized dictionaries 92, 93, 94 text messaging see txt language
speech-recognition software 8 Tidwell, J. 123
spelling: fan subtitling 153; lexicographic TM see translation memory
needs and preferences 91; subtitling Tobii X120 157
152; txt language 6, 151–2, 154, 155 tokens 37, 155–6
Star Transit 116 TQA see translation quality assessment
status 59–60 Trados Studio 15, 23n5, 65, 122–4; see
Steyn, F. 159 also SDL Trados
STS see science and technology studies trainers 18–19
students: attitudes to technology 18–19; training 141, 142
lexicographic needs and preferences 86, Transl8it! 155, 167n1
87; ‘love letters’ 11, 13, 14, 15–18 Translation Bureau (Canada) 58
subjectivity 131–2, 134, 135, 137, 139, 141 translation edit rate (TER) 134
subtitles 6, 149–50, 152–67 translation memory (TM) 3, 8, 13, 19,
Surfeurs du Paradis 153 23n4, 45–6; benefits and drawbacks of
surveys 4; post-editing user interfaces 5, 48; Facebook translation crowdsourcing
109–10, 112–13, 114–22, 126; termino- 36; implementation of 4–5, 48–9; loss of
lexicographic needs and preferences autonomy 28, 55–6; machine translation
81, 83–97, 104; trainers’ and students’ research 135; ‘mangle of practice’
attitudes to technology 18–19; concept 64–5; post-editing 109, 110–11,
translators’ emotions 11; txt language 113, 115–17, 120–1; productivity
comprehension 165 requirements 49–50, 51–3, 56; recycling
syntagmatic text processing 65, 72, 75–6 of existing translations 50–1, 53–5;
research methodology 46–7; translation
TagEditor 123 studies 2; translators’ letters 14, 15; trust
Tagg corpus 155 29; use of 47–8
Tagliamonte, S. A. 154 translation quality assessment (TQA) 5–6,
Taravella, A. 58–9 131–48; evaluation materials 142–3;
target languages 115–16, 119 evaluators 140–2; in industry 136–7;
Tarp, S. 80, 81 machine translation research 133–6;
technology: acceptance of 9–10, psychometrics 138; reliability 139–40;
23; autonomy of translators 40; translation studies 132–3; validity 138–9
crowdsourcing 26, 37; democratizing translation service providers (TSPs) 4–5,
nature of 3; English-Chinese translation 45, 46–7, 58–60; implementation of
web resources 74; human aspects 58–9; TMs 48–9; productivity requirements
‘pro-innovation bias’ 1; proliferation of 49–50, 51–3, 56; recycling of existing
translation tools 45; technologization of translations 50–1, 53–5; use of
Index 177
translation technologies 47–8; see also translation memory systems 111;
language service providers translation quality assessment 143n1
translation studies 2, 30–1, 81, 132–3 usefulness 9; MeteoTrad 101–2; post-editing
translators: autonomy 27–8, 40, 45, 55–6, user interfaces 119
60; decline in professional satisfaction user-centred design (UCD) 111
57, 60; emotions and attitudes to user experiences 10, 19–22
technology 4, 8–23; Facebook user-generated content (UGC) 25, 27
translation crowdsourcing 35, 36; user interfaces (UIs) 5, 109–30; Facebook
function theory of lexicography 82, translation crowdsourcing 33, 36;
83; ‘human intelligence’ 31; impact interview results 122–5; lexicographic
of translation memory systems 4–5, tools 83; research design 112–14; survey
45–6, 47–51, 58–9; information needs results 114–22; translation quality
and preferences 5, 80–1, 83–97, 100–1, assessment 137
104; MeteoTrad 97–104; paradigmatic users 2–3, 5, 111
text processing 65, 71–2, 75–6;
post-editing user interfaces 109–10, validity 63, 89, 132, 133, 137–9, 142,
112–25; quality assurance 38–9; 143n2
reaction to crowdsourcing 25–6, 27; Vermeer, H. J. 81
shifts in practices 45–6, 51–7, 58–9; Villeneuve, A. O. 58–9
status of 59–60; translation quality vocabulary 151, 155
assessment 137, 141; usability 19–22;
web resources 5, 63, 64, 65–76; see also Wang, Vincent X. 2, 3, 5, 63–79
junior translators; senior translators Web 2.0 25, 27, 30, 32, 38, 39
Translators for Ethical Business Practices web resources see Internet
27 weighted calculations 49–50, 51–3
TransStudio 116 Werry, C. C. 152
TransType 111 Wikipedia 2, 70
trust 4, 26, 28–9, 38–9, 40 Windhager, F. 40
TSPs see translation service providers ‘wisdom of the crowd’ 29
TTR see type token ratio WMT see Workshop in Machine
Turchi, M. 140 Translation
Twitter 27, 152 Wong, Y. 26
txt language 6, 149–70; comprehension word frequency 154–5, 158
165; data analysis 158–9; description word-order changes 111, 119, 124
of 150–2; facilitation effect 163–5; word processing 8, 19
rationale for using in subtitling 152–3; Wordfast 118, 123
research methodology 153–8; research Wordreference.com 92
results 159–65 workplaces 46–7
type token ratio (TTR) 155–6 Workshop in Machine Translation (WMT)
136, 139, 140
UCD see user-centred design Wotschack, C. 159
UGC see user-generated content
UIs see user interfaces XTM 116, 119
ukWaC 155
uncertainties 83, 84 Youdao dictionary 63, 68, 69, 70
United Kingdom 150
usability 4, 9–10, 11, 12, 19–22, Zaidan, O. F. 30
23; criticism of technology 14; Zhao, Y. 25
lexicographic needs and preferences Zhongguo Yidian 63
86; post-editing user interfaces 116–17; Zhu, Q. 25

Human Issues in Translation Technology

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Human Issues in Translation Technology

Uploaded by

Copyright:

Available Formats

Human Issues in Translation

Dorothy Kenny is Associate Professor in the School of Applied Language and

The International Association for Translation and Intercultural Studies (IATIS)

Human Issues in Translation Technology

Edited by Dorothy Kenny

ISBN: 978-1-138-12329-8 (hbk)

Typeset in Times New Roman

1 Love letters or hate mail? Translators’ technology acceptance

2 Deconstructing translation crowdsourcing with the case of a

3 ‘I can’t get no satisfaction!’ Should we blame translation

4 How do translators use web resources? Evidence from

5 Translators’ needs and preferences in the design of

7 Issues in human and automatic translation

8 Cn U read ths? The reception of txt language in subtitling 149

1.1 Usability and positive affects 20

1.1 All letters by respondents 11

Stephen Doherty is a senior lecturer in the School of Humanities and Languages at

The International Association for Translation and Intercultural Studies (IATIS)

I am deeply indebted to all of the contributors and anonymous peer reviewers

Human Issues in Translation Technology is a coy title for a book. It seems

Emotions and technology acceptance

[F]ew if any perceptions of any object or event, actually present or recalled

More specifically, emotions also play a major role in acceptance of new

Research methodology and the respondents

Love letter Ambivalent letter Break-up letter Total

Or do some reverse thinking: What is the most annoying hindrance you

• 44 professional translators in the European Union (EU) institutions;

. . . You do not, however, need to focus on specific translation tools or

(Translation) Technology as an object of love and hate

Love letter Ambivalent letter Break-up letter Total

Table 1.3 Letters commenting on translation-specific tools

Love letter Ambivalent letter Break-up letter Total

EU/FI translators MA students

sometimes claimed. Of the respondents who commented on translation-specific

Time, experience and technological change

Work experience Respondents Love Ambivalent Break-up

Junior translator = < 10 years 20 (10 EU + 10 FI) 14 1 5

Usability and user experience

Learnability System errors

Figure 1.1 Usability and positive affects

belong to this category and express sentiments of (dis)satisfaction. In this analysis,

Learnability System errors

Figure 1.2 Usability and negative affects

Discussion and conclusions

Research on crowdsourcing and translation crowdsourcing

Understanding the role of technology and relations

Unpacking Facebook Translation crowdsourcing according to ANT

Figure 2.1 A potential network of actors linked to the FTC platform

this analysis, Facebook management is represented as the Facebook FTC team

Figure 2.2 Facebook Translations platform configuration

and voting on input by other FTC contributors. Importantly, Facebook also

Workplaces and methodology

The translation service providers

TSP-A employs 55–60 employees, 28 of whom are language professionals

The use of translation technologies in the TSPs under study

Benefits and drawbacks of TMs: an overview

• help to increase productivity;

The main disadvantages, on the other hand, were that:

Implementation of TMs and the establishment of new guidelines

TSP-B and TSP-C

Enforced recycling of existing translations

Translators’ perceptions of shifting practices

Productivity requirements and weighted calculations

This view is echoed by a beginner translator at TSP-C: