See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/316878404

Review of Carl, Michael, Srinivas Bangalore & Moritz Schaeffer, eds (2016)
New Directions in Empirical Translation Process Research: Exploring the
CRITT TPR-DB

Article  in  Babel · July 2017

DOI: 10.1075/babel.63.3.12yua

CITATIONS READS

0 37

2 authors, including:

Yu Yuan
University of Leeds
8 PUBLICATIONS   4 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Java Automatic Term Extraction toolkit (JATE) View project

text mining View project

All content following this page was uploaded by Yu Yuan on 14 November 2017.

The user has requested enhancement of the downloaded file.

 informations bibliographiques
et lexicographiques
bibliographical and
lexicographical information

Michael Carl, Srinivas Bangalore and Moritz Schaeffer (eds). New

Directions in Empirical Translation Process Research: Exploring the CRITT
TPR-DB. Heidelberg, New York and London: Springer, 2016. 315 pp

Reviewed by Yu Yuan & Serge Sharoff

Nanjing University of Information Science and Technology, School of
Languages and Cultures, Nanjing University of Information Science and
Technology, 210044 Nanjing, China
Centre for Translation Studies, University of Leeds, LS2 9JT, Leeds,
United Kingdom
E-mail: hittle.yuan@gmail.com

Translation process research (TPR) can be traced back to decades ago and only
in the last decade or so, several volumes of collected work on translation ­process
have been published (Saldanha and O’Brien 2014: 109). In recent years, the ­written
translation process appears to have attracted increasing attention as a direct result of
researchers’ using new research tools and technological e­volution in the translation
profession. Unlike in the early days it was verbal data from think-aloud ­protocols
(TAPs) that were mainly explored, more recently, researchers of ­translation pro-
cess have taken a more data-based empirical approach that incorporates into their
research lab experiments. While studies using retrospective and introspective data
have been highly valuable in the investigation of the cognitive ­processes during
translation, the very act of verbalising thoughts has been c­ riticized for interfering
translators’/interpreters’ cognitive efforts in work, degenerating in accuracy and
unavailability for reuse and verification. In the p
­ resent ­volume, a series of research
on diverse directions in TPR centring around the CRITT T ­ PR-DB are described
as the latest developments in TPR.
The book is clearly structured with three parts. The first two chapters serve
as the foundation for the ensuing empirical TPR studies. Chapter 1 situates the
TPR research in the prescriptive-predictive shift within the broader framework of

Babel 63: 3 (2017), 450–456.  © Fédération des Traducteurs (fit) Revue Babel
doi 10.1075/babel.63.3.12yua  issn 0521–9744  e-issn 1569–9668
 Bibliographical and lexicographical information 

descriptive translation studies, bringing to notice the latest developments in mod-

ern TPR and then in the second half a brief summary is given of each ­following
chapter, followed by Chapter 2 introducing the CRITT TPR database that ­contains
“user activity data (UAD) of translators, editors, post-editors and authors’ behav-
iour recorded with Translog-II and the CASMACAT workbench” (Carl, Bangalore,
and Schaeffer 2016: 7). The database consists of tables and features of recorded
translation sessions. Typical in these tables and among the features are basic and
composed product and process unit information, such as editing activity (number
of insertions and deletions), parallel reading activity, temporal information of ses-
sions keystroking data, fixation data recorded by an eye-tracking device, keylog-
ging of external resource.
Part II and Part III are made up of 12 remaining chapters that are studies
targeting (a) specific research question(s) with the facilitation of TPR-DB data.
These individual studies delineate a diverse directions of computational, statisti-
cal and psycholinguistic modelling in the translation process. ­Chapter  3 inves-
tigates the effects on translation productivity of integration of online (OL) and
active learning (AL) features into the computer-assisted translation w ­ orkbench
­CASMACAT. In this chapter, the mathematical theorems behind the statistical
approach to machine translation and the statistical interactive translation predic-
tion (ITP) are first briefly described, followed by an introduction to online and
active learning for statistical machine translation systems (SMT) and experiments
under three different setups, i.e., conventional ITP, ITP with online learning and
ITP with active learning, are designed to compare and assess online learning and
active learning influence on translators against the two measures of speed and
effort for translation process. Results show that ITP systems incorporating OL
required less typing effort and had increased the speed for 60% of the translators,
and that ITP system with AL obtained consistently better translation quality than
the conventional ITP, but the difference in translation speed between ITP with AL
and conventional ITP is insignificant due to the possible fact that translators tend
to double check the translations generated by the updated systems.
Chapter  4 looks into the impact of interactive machine translation on
­post-editing using metrics proposed by Krings (2001) to quantify temporal,
­technical and cognitive effort required to post-edit two specialized texts.The post-
editing effort measured is then correlated with the computed translation edit rate
(TER) scores in order to test two hypotheses that “interactivity will c­ ontribute to a
­significant decrease in the time spent on post-editing tasks” and “TER scores will
be lower for ITP post-editing compared to standard PE” (Alves et al. 2016: 79).
Results in their study have shown that subjects neither became faster nor showed
a decrease of the number keystrokes when working in the ITP ­post-editing
 Informations bibliographiques et lexicographiques

e­ nvironment, but ITP does have a positive impact on cognitive effort as evidenced
by the considerably shorter fixation duration.
Aiming to investigate whether exposure to the ITP working mode over a lon-
ger period of time has an effect on translators’ post-editing behaviour, Chapter 5
conducted a longitudinal study (LS14) to compare translators’ performance when
they are working with traditional post-editing mode and the ITP mode. Results
show that participants became indeed faster over the period of working with the
ITP workbench. In the same study, behavioural patterns of LS14 participants are
compared with patterns of those who have no previous ITP working experience
in another non-longitudinal experiment in order to assess what post-editors have
learned during the process of working with the ITP mode. As measured by the
relative proportion of coherent keystroke activities to the filtered total production
duration, post-editors overwhelmingly seem to have learned to accept interac-
tive suggestions which help to reduce the amount of their coherent typing time.
­However, in a follow-up user feedback, most translators offer a positive view on
ITP and state that they would favour the ITP workbench but there is a clear cut
­difference between the extremely positive attitude towards ITP shown by the less
experienced translators and the negative by more experienced ones.
Consulting external resources is an inevitable aspect of the translation pro-
cess. With the data captured by the integrated functionalities of ITP workbench
and Inputlog Chapter 6 analyzed the usage of external resources on productivity
and quality of final product from the two aspects of acceptability and adequacy.
Through the linear mixed effect modelling, it is confirmed that significantly more
time is spent on external resources in translation than in post-editing and no sig-
nificant difference could be found in the types of resources consulted in both tasks.
In a similar vein, another mixed effect model with random effects is fit to explore
the effect of consulting external resources in translation and post-editing on qual-
ity. As far as post-editing is concerned, spending a longer time in querying external
resources does not guarantee an increased quality but in the case of ­translation
extra efforts to consult external resource do pay off, leading to the decrease in over-
all error score. Statistical tests show that there is no significant difference between
human translation and post-editing but the specific type of external resources does
have an individual difference. Consulting dictionaries can bring about increase in
acceptability but using encyclopedias, out of expectation, gives rise to an increase
in error score. Further on this, Chapter 7 investigates how translators interact with
the biconcordancer (BiConc) on the workbench and other informational resources
in task-specific post-editing. The author argues that such type of research is infor-
mative to workbench development and future web-based translation environments.
Chapter 8 to 14 make up Part Three of this volume that deals with the topic of
translation behaviour modelling. In Chapter 8, a statistical method to a­ utomatically
 Bibliographical and lexicographical information 

tagging raw keylogging and eye-tracking data into distinct human ­translation pro-
cesses (HTPs), such as orientation, revision or pause. This is indeed a machine
learning task by segmenting the recorded translation sessions into consecutive
windows that form observations of feature vectors, and a Hidden Markov Model
(HMM) is then trained with the automatically clustered labels from the raw data
as the predicted output labels, on the assumption that the probability of one HTP
next is conditionally determined by the current HTP. With this HMM model, the
researchers use the Viterbi algorithm to label the HTP data automatically. For the
purpose of validation, the performance of the tagger is then benchmarked with
manually annotated data by human experts in TPR. The results suggest that not
only does this automatic annotation method demonstrate as high accuracy as
human experts, but the unsupervised way of discovering process labels through
clustering displays good validity.
On the hypothesis of co-activation of bilingual systems during ST reading,
Chapter nine examines “the effect of cross-linguistic syntactic re-­ordering and
word translation entropy on early and late eye movement measures in translation”
(Schaeffer et al. 2016: 195). Results support the hypothesis that translation entropy
(the number of translation alternatives) and differences of word order in source
and target text have an effect on early and late eye movement. The authors believe
the findings can be explained by semantic and structural c­ ross-linguistic priming,
i.e., items that are similar in word order and syntactic structures are more likely
to become the candidate translations, and for the same reason items that have few
translation alternatives are more likely to prime too, a circumstantial evidence for
the tendency of literality.
In view of syntactic priming, Chapter  10 tests the hypothesis that priming
effects go beyond the lexical level with translation data of three language pairs.
After a brief introduction to the concept of translation entropy, a measure of
­translation effortfulness and a description of translation variation, the authors
posit the concept of syntactic entropy as a measure of syntactic uncertainty for
a translator to choose from random target syntactic structures and correlate this
measure with data collected in a monolingual copying task and the TPR data-
base. The findings justify the hypothesis that priming effects of shared linguis-
tic represenations extend to syntactic level as is manifested in longer behavioural
measures, indicating the selection pressure that is brought about by the multiple
syntactic ­equivalents increasing the cognitive load for the translators.
Instead of working on closely related language pairs, Chapter 11 is an explor-
atory study of the impact of the type of cohesive chains on the cognitive effort
in P­ortuguese-Chinese during translation and post-editing tasks. Two types of
­cohesive chains are analyzed with eye movements and keyboard data in order
to find out if a significant cognitive difference exists to understand and produce
 Informations bibliographiques et lexicographiques

two cohesive chains and if processing cohesive chains differs in different work-
ing scenarios, such as translation and post-editing. In their findings, the authors
assert that type of cohesive chains is crucial to impact eye movements and posit
that producing cohesive chains in target texts is more cognitively challenging.
Other than this, they also claim that type of task has an effect on dependent vari-
ables such as total reading time, number of fixations and first pass duration on
ST and TT. This study is characteristic of attempting to deal with the logographic
language like Chinese in the process of collecting eye-tracking and keystroking
data, too.
Chapter 12 resembles the study in Chapter 10 in that it deals with the cogni-
tive process of structure choice in the target language. However, this chapter is
more like a review of the previous studies than a complete, self-contained research.
The author first reviews measures of cognitive load in translation and then elabo-
rates on whether the correlation between translation entropy and reading times
can carry over into structure choice, which is critically discussed from the aspects
of syntactic, functional and conceptual annotation systems respectively. The mer-
its and demerits of each annotation system for structure choice are highlighted.
Under the assumption that literal translation is the default procedure used by
both novice and experienced translators, in Chapter 13, Kielar’s definition of liter-
ality (2013: 51) that calqued words from source language are combined to be sepa-
rate lexical units in target language is borrowed to operationalize literality, a study
is designed to ascertain whether translators of different experience or proficiencies
differ. Three groups (non-professional bilinguals, student translators and profes-
sional translator) of participants are mixed to be compared as control and obser-
vation groups in a French-Polish translation, and multiple analysis of variance
showed experience plays a significant role in producing literality in translation as
students tend to translate less literally than professional translators. However, no
significant differences can be identified in the number of units translated literally
between professionals and non-professionals. As far as the pre-translation mental
representation of the source text is concerned, it impacts disparately on the three
groups: students who did a mental representation of the source text before the
translation tend to translate literally and in contrast professionals translate more
freely. The results in the study also verify the hypothesis that words in one lan-
guage have a limited number of literal translation equivalents in another.
Chapter 14 introduces a new framework of categorising post-editing behav-
iour into five classes and compares the differences in translation and post-editing
in different domains, using the quantified key-logging (production time and pro-
duction activity) and eye-tracking (reading time) data. According to the authors,
the results confirm that the time for post-editing is shorter than for translation
irrespective of the text domains but translation of domain-specific texts requires
 Bibliographical and lexicographical information 

more efforts. This study highlights the importance of a richer annotation schema
for translation process and that including domain as a variable in the study offers
better insight into how texts from different domains are processed in the transla-
tion and/or post-editing process.
One of the most commendable achievements of this volume is that it
opens a new venue for TPR by taking an empirical data-driven approach. More
­specifically, in comparison to those subjective and incomplete introspective and
extrospective data elicited by whatever means, keystroking and eye-tracking
data are more reliable resources of representing the cognitive load of transla-
tors and post-editors in the process of translation and post-editing. Common
in this collected work, many automatically computed variables such as parallel
reading, fixation and activity units, in addition to those manually annotated data
such as word and syntax translation entropy, are incorporated into the rigorous
research designs that are characterized by robust statistical modelling, such as
linear mixed effects models, which depart from the conventional subjective inter-
pretation of elicitation data. Moreover, various topics are covered in this volume
investigating translation ­process. Such topics range from working mode differ-
ence (interactive v.s. traditional), longitudinal influence (system exposure v.s no
system exposure), external resource impact (dictionary, corpus, biconcordancer
and so on), automatic classification of translation process, lexical and syntactic
variance on translation and cohesive relations in text comprehension to literality.
This small set of data- and lab-based research could provide food for thought for
future research directions in TPR. Nevertheless, it is questionable as to whether
or not the book should be organized with the current structure. For example, Part
I, though as the title of empirical TPR suggests, seems to be devoted to defining
for the readers the field of empirical translation process research as a lead-in sec-
tion. To our surprise, not much has been mentioned and only a few paragraphs
are spent on the development of this field but great efforts have been made to
summarize the following chapters. Perhaps, it would be better to have a more
extended introduction to the latest developments of empirical TPR in a single
chapter with a coherent presentation of major concepts, mainstream methodol-
ogy and other essential background information. Similarly, why Part II is named
post-editing with CASMACAT while apparently more than one study included in
this section is about both translation and post-editing? Apart from this imperfect
presentation, some of those issues are associated with the methodology of specific
research. First, studies involving computational variables (e.g., Chapter 3, Chap-
ter 9 and C
­ hapter 10) did not give a practical implementation of computing those
variables, such as translation entropy, but instead chose to explain them in the
daunting mathematics for many ­translation scholars. This negligence constitutes
an obstacle to replicate these variables in other research to follow. In addition,
 Informations bibliographiques et lexicographiques

several studies in this book use heavily the manually annotated data for analysis
but no reports of i­nterannotator agreement whatsoever are given. The missing
information of the number of annotators and interannotator agreement could
affect the validity of such data and as a consequence could potentially jeopardise
the reliability of any findings from the research.
On a final note, we must admit this book is a useful resource to the new fron-
tiers of empirical translation process research. It should serve a positive reference
for translation teachers and researchers, as well as graduate students, who want to
be acquainted with new ideas in this subdiscipline and undertake in-depth studies
in other new directions.

References

Alves, F. 2016. “Analysing the Impact of Interactive Machine Translation on Post- editing Effort”.
In New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB.,
ed. by M. Carl, S. Bangalore and M. Schaeffer, 77–94. Heidelberg – New York and London:
Springer.  doi: 10.1007/978-3-319-20358-4_4
Kielar, B. Z. 2013. Zarys translatoryki. Warsaw: Wydawnictwo Naukowe IKL@.
Krings, H. P. 2001. Repairing Texts:Empirical investigations of machine translation post- editing
processes. 1st ed. Kent – Ohio and London: The Kent State University Press.
Saldanha, G. and O’Brien, S. 2014. Research Methodologies in Translation Studies. London – New
York: Routledge.
Schaeffer, M. 2016. “Word Translation Entropy: Evidence of Early Target Language Activa-
tion During Reading for Translation”. In New Directions in Empirical Translation Process
Research: Exploring the CRITT TPR-DB., ed. by M. Carl, S. Bangalore and M. Schaeffer,
183–210. Heidelberg – New York and London: Springer.  doi: 10.1007/978-3-319-20358-4_9

View publication stats