See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/264382314

EEG, EYE and KEY: Three Simultaneous Streams of Data for Investigating the
Cognitive Mechanisms of Translation

Chapter · January 2011

CITATIONS READS

15 666

1 author:

Christian Michel Lachaud

University of Oslo
15 PUBLICATIONS   174 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Christian Michel Lachaud on 12 February 2016.

The user has requested enhancement of the downloaded file.

 EEG, EYE and KEY: Three Simultaneous Streams of Data for Investigating the

Cognitive Mechanisms of Translation

Christian Michel Lachaud

I.S.T. European Research Project “EYE-to-IT”,

Department of Literature, Area Studies and European Languages,

University of Oslo, Norway

Introduction
The psycholinguistic study presented in this article was realized for the European

Research Project “EYE-to-IT”,1 in collaboration with the National Centre for Epilepsy,

Rikshospitalet University Hospital, Oslo, Norway. Using an original technical

development combining three technologies, namely electroencephalography (EEG),

eye tracking (EYE) and keystroke logging (KEY), EYE-to-IT aimed to develop

technical solutions for studying the cognitive mechanisms of translation and for

helping translators to perform better while translating texts.

“Prompting,” or presenting online a visual message composed of a word or a

sentence, offers interesting perspectives for the purpose of supporting a translator in

his task. The prompting technique, already in use for online documentation and

feedback systems (Cavero et al., 1995; Ferreira & Atkinson, 2009), could be adapted

for priming transcoding and translation (Duyck & Warlop, 2009; Finkbeiner et al.,

2004) - “transcoding” being a technical term from Translatology, meaning “translating

out of context” (Lederer, 2006). Because a successful prompting procedure should

not restrict the translator’s freedom, fluidity and creativity by locking a given

formulation in his mind, prompting should not consist in delivering a list of possibilities

1
from which the translator would choose the one he prefers. Rather than providing a

panel of alternative translation possibilities, prompting should influence the

translators’ mental processes, supporting them when they fail or are being

challenged. Prompting can only achieve this goal by offering a fuzzy, even subliminal

influence, using prompts different enough from the targeted solution to avoid focusing

the translator on a formulation, but related enough to support translation. One

solution is to prompt formal and semantic representations via formal and semantic

features, respectively letters or sequences of letters and/or semantic relatedness. For

instance, the Norwegian word “mannekeng” (fashion model) can be used to prompt

the Norwegian transcoding of the English source word “male”, the target word

“mannlig”, because both prompt and target share sublexical features (i.e. the first four

letters). Similarly, a semantic prompt can be found in “bestefar” (“grandpa”), which is

a specific representative for the category “male”. With such inputs, the translator

would be left dealing with his own mental processes, the dynamics of which are now

subtly influenced.

Although we acknowledge that translation is carried out at the textual level, a

systematic approach intended for future technical implementation requires that the

positive effect of this prompting method on translation is evaluated step by step,

starting at the word level before using more complex linguistic stimuli in experimental

settings, like phrases, sentences and texts. The study presented in this article

involved the combining of EEG, EYE and KEY for investigating cognitive loading

when transcoding Deceptive Cognates, True Cognates and Non-Cognates (see

below for an explanation of each of these). Our objective was to be able to provide

helpful prompts to translators, based on what we could find about cognitive load in

these three possible transcoding scenarios. We will start by discussing the

2
transcoding challenge existing in each situation. We will then discuss the advantages

of each measurement technique (EEG, EYE, KEY), as well as the advantages and

disadvantages in combining them. Finally, we will describe the method, present our

results, and discuss the findings.

“Back-engineering” the mind of bilinguals

The experiment presented in this article was designed for identifying transcoding

challenges and for starting to explore the psychological nature of these challenges. It

was the first in a series of four experiments which were designed for (i) identifying the

transcoding challenges that a translator faces, (ii) understanding the psychological

nature of these challenges, (iii) defining the type of information that could helpfully

prompt the translator in order to cope with transcoding challenges, and (iv) evaluating

the benefit of prompting for transcoding performance. Three logical L 2 (English) - L1

(Norwegian) word correspondences were used in the study: Deceptive Cognates or

false friends (DC), True Cognates (TC) and Non-cognates (NC) (Chamizo

Domínguez & Nerlich, 2002).

(i) A DC relation exists between two words from two languages when they share

the same form but not the same meaning, like “gift”, meaning “present” in

English, but “poison” in Norwegian.

(ii) A TC relation exists between two words from two languages when they share

both the exact same form (contrary to cognates whose form may differ slightly

in a language pair) and the same meaning, like “egg” for the

English/Norwegian pair.

(iii) A NC relation exists between two words from two languages when they share

the same meaning but not the same form, like “ant” in English and “maur” in

Norwegian.

3
 Figure 1 shows the logical structure of L2-L1 correspondences at the lexical

and semantic levels for DC, TC and NC, based on the architecture of Conceptual

Mediation Hierarchical Models (French & Jacquet, 2004; Frenck & Pynte, 1987;

Schwanenflugel & Rey, 1986).2

2 1 2/1 2/1

Semantic
Lexical

C2 S2/1 C1 S/C2/1 S2 C1

L2 L1 L2 L1 L2 L1

Deceptive Cognate True Cognate Non-cognate

Figure 1. Logical structure of lexical and semantic L2-L1 relations for Deceptive

Cognates, True Cognates and Non-cognates. S stands for “Stimulus” (English word

to transcode into Norwegian), C for “Correspondence in the other language”, L for

Language; L2 is the second language of the subject (English); L1 is his mother

tongue (Norwegian), into which he must transcode. The numbers 1 and 2 refer

respectively to L1 and L2. Squares represent the mental representations of lexical

information. Circles represent the mental representations of abstract information.

NC corresponds to the most common transcoding situation, while in

comparison, TC and DC are rare. TC can be considered as a control condition in

which no transcoding needs to be done, while DC and NC require the application of a

4
transcoding procedure. In theory, transcoding an English NC Stimulus S 2 like “ant”

into its Norwegian correspondence C1 “maur” will require four main steps: (i)

recognizing the stimulus (retrieving and activating in long-term memory the mental

representation of its formal aspects – symbolised by the square marked S2 in Figure

1), (ii) accessing the meaning (activating in long-term memory the mental

representation of the abstract aspects of the stimulus – symbolised by the circle

marked 2/1 in Figure 1), (iii) retrieving the correspondence in the target language

(activating in long-term memory the corresponding formal representation for

Norwegian, marked C1 in Figure 1), and (iv) producing the solution (speaking, typing,

writing). In the case of TCs, because both the formal and the semantic

representations are identical in L1 and L2, no transcoding will happen. Therefore, the

processing of a TC stimulus will be limited to three steps: (i) recognizing the stimulus,

(ii) accessing its meaning, and (iii) producing the solution. Transcoding English DC

input S2 (like “gift”) into its Norwegian correspondence C1 (“gave”) will require

additional processes, since S2 is conflated with S1 (“gift” can be an English word as

well as a Norwegian word). According to parallel distributed processing models

(Group, 1986), whose principles are well formulated in TRACE (McClelland & Elman,

1986), a psycholinguistics model of word recognition, the stimulus will not only

activate meaning 2 (“present”) in the English lexicon, it will also activate meaning 1

(“poison”) in the Norwegian lexicon (see also (Bölte & Coenen, 2002). Activation will

automatically propagate in the neural network, spreading back from the semantic

level of representation to the formal level of representation in the other language

(Figure 1). Therefore, two transcoded meanings of “gift” will be simultaneously

activated, one in Norwegian from the English meaning (C1 “gave”), and one in

English from the Norwegian meaning (C2 “poison”). The consequence will be a

5
conflict between the two lexicons during selection, both at the semantic level between

meanings 1 and 2 and at the lexical level between representations C1 and C2. Facing

a confusing situation, the translator may have four types of reactions: (i) He might

automatically produce the correct Norwegian formulation (“gave”) for the English

meaning (“gift”), although probably with greater difficulty than for a NC; (ii) Confused,

he might also produce an incorrect output by formulating the Norwegian meaning in

English (“poison”), especially if the Norwegian language is more prominent in his

mind than the English language. He may then realize his mistake and try to correct it,

in which case two scenarios may occur (iii and iv). A successful correction would

require him (iii) to go back from C2 (“poison”) to meaning 1 (“poison”), to S2/1 (“gift”),

and to proceed forward in the other direction, to meaning 2 (“gift”), to C1 (“gave”). If

successful, the time needed to produce the correct Norwegian transcoding (“gave”) of

the English stimulus (“gift”) will be further increased and may have other behavioural

manifestations, like typing corrections. But the translator may also be unable to fully

correct his mistake, especially under time pressure, (iv) if he only reverses his

transcoding, and does not transcode again. In this case, he will simply be back to the

initial stimulus (“gift”), a phenomenon that was sometimes observed in our

experiments.

Measuring the mind simultaneously with EEG, EYE and

KEY
While the mind cannot be measured directly, manifestations of its activity can,

such as electric potentials (EEG) and behaviour (eye movements and keystrokes).

The EEG technique measures, usually on the surface of the scalp, the electric

variations generated by the activity of neuron assemblies (Niedermeyer & Da Silva,

2005), including that of cognitive activity, at the moment they occur in the cerebral

6
cortex (“online” measurement). EEG can be used to track cognitive load variations as

an index of processing difficulty, to locate where in the brain the activity occurs for

additional knowledge about the cognitive functions involved, or to pinpoint the

moment in time when a specific process occurs. The eye-tracking technique

measures eye movements (also considered to be an “online” measurement), a

behaviour which provides indices about visual information intake. Eye movements

can reveal the progression of syntactic processing and meaning building in the

reader’s mind, and in the cases where information intake is necessary for the

translation process, or in the cases where the translation processes influence

information intake, eye-tracking may also reveal information about the cognitive

processes occurring during translation. Finally, keystroke logging, the technique of

recording typing behaviour on a keyboard, can be used for studying the mental

processes during translation by studying how the text was produced (also considered

“online” measurement) as well as for studying translation as a product (offline

measurement), in which case keystroke logging only serves as a means to record the

final text product.

Despite the fact that each technique imposes its own constraints on the

experimental situation and is generally seen as exclusive of the two others, EYE-to-IT

engaged researchers in the challenge of simultaneously using EEG, EYE and KEY

for studying cognition during translation. By multiplying measurement techniques, a

tremendous burden is placed on researchers to control their experiments and to

prepare the participants adequately. Furthermore, the risk of producing bad quality

data is increased. However, combining techniques offers the invaluable advantage of

yielding a thorough picture of the reality being measured.

7
 The main technical difficulty in this study was presented by EEG and EYE.

EEG requires situations which do not promote eye movements, blinks, motor activity,

and any cerebral activity other than the one being studied. Therefore, reading and

typing while translating represent sources of artefacts that one would like to avoid,

making it meaningless not only to use EYE and KEY together with EEG, but to use

EEG for studying translation. However, the improvement of mathematical algorithms

allows us to extract useful information from EEG with artefacts, opening its

application to new research areas. Similarly, using eye-tracking in a situation where

the subject regularly gazes beyond the screen (typing, consulting a dictionary) is a

challenge in itself, as the tracking of the eyes’ position will be interrupted.

Furthermore, it may be difficult, if not impossible, to distinguish, during the analysis of

eye movements, between actual reading activity and the search behaviour that

occurred when the translator transferred attention back to the source text. However,

eye-tracking proves to be at least a very useful tool for the online detection of the

difficulties that the translator is facing (longer fixation time or larger number of

fixations). Subsequently, eye-tracking provides inputs for analyzing the nature of the

linguistic challenges and knowing when to deliver a prompt and where on the screen.

If constraints can be managed, each technique measuring a different manifestation of

the mental processes underlying translation may be used in a beneficial and

complementary way with other techniques. This requires clever designs and proper

methods of analysis for reducing the impact of artefacts on experimental results.

In this study, we investigated (i) the amplitude variations through time in the

EEG frequency bands (Time-Frequency Power Differences), which reveal cognitive

load variations, (ii) the verification effort during the mental matching between L 2 and

L1, which manifests in the total amount of time spent gazing at both words (Total

8
Fixation Duration), (iii) the attention demand required for processing the stimulus, the

variations of which are manifested in pupil diameter through the pupillary reflex (Pupil

Size Amplitude Variation, corresponding to the difference between minimum and

maximum sizes), (iv) the decision difficulty, which is manifested in the amount of time

required before pressing the answer key (Reaction Time), and (v) the confusion of

the translator, which is manifested in the amount of errors (Error Probability).

We expected that TC would be the easiest configuration to process (lowest

cognitive load, verification effort, attention demand, difficulty and confusion), while

DC would be the most difficult configuration to transcode (highest cognitive load

increase, verification effort, attention demand, difficulty and confusion). Non-cognates

were expected to present intermediate levels of difficulty.

Method
Independent variables
Two independent variables were manipulated in the experiment:

(i) Stimulus Type (three levels: DC, TC, NC)

(ii) Transcoding Correspondence between the L2 and L1 words composing

a stimulus (two levels: Match - L2 and L1 correspond, Mismatch - L2 and

L1 do not correspond).

The two variables were crossed according to a factorial design, generating six

experimental conditions.

Dependent variables
The dependent variables, measured while both the English and the Norwegian words

were displayed, were as follows:

9
 (i) Time-Frequency Power Difference (EEG) between the Match and

Mismatch conditions for each stimulus type (expressed in significant

Time-Frequency units - see the Data Processing section).

(ii) Total Fixation Duration (expressed in ms - EYE)

(iii) Pupil Size Amplitude Variation (expressed in arbitrary unit3 - EYE)

(iv) Reaction Time (expressed in ms - KEY)

(v) Error Probability (expressed in per cent - KEY)

Operational Hypothesis
EEG: Time-Frequency Power Difference will be positive with DCs (greater cognitive

load for processing the Match condition than the Mismatch condition), negative with

TCs (smaller cognitive load for processing the Match condition than the Mismatch

condition) and intermediate with NCs.

EYE: decreasing gradient of Total Fixation Duration and Pupil Size Amplitude

Variation will be observed from DCs, to NCs, to TCs.

KEY: decreasing gradient of Reaction Time and Error Probability will be

observed from DCs, to NCs, to TCs.

Experimental materials
120 English words and 240 Norwegian words were used. The English word list

consisted of 40 English/Norwegian DCs like “art” (“art” means “sort, kind, or species”

in Norwegian), 40 English/Norwegian TCs like “arm” and 40 English/Norwegian NCs

like “ant”. 120 of the 240 Norwegian words were the Norwegian correspondence of

the English words (respectively “kunst”, “arm” and “maur”), the other 120 being

unrelated. 240 items were created by associating each one of the 120 English words

once with its Norwegian correspondence (120 items) and once with one of the 120

unrelated Norwegian (120 items). The stimulus words were all nouns. In order to

10
avoid ambiguous situations, NC words were excluded if they could be a cognate with

a slightly different spelling (like porter/portier, sister/søster, hair/hår, machine/maskin,

etc.). Spelling in English and Norwegian were strictly identical for TCs and DCs. As

shown in Table 1, the average length of the English words was approximately

identical across Stimulus Type conditions, as well as the Thorndike-Lorge written

frequency index of word occurrence in written language (Wilson, 1988). Other indices

known to influence processing difficulty were also computed: Age of Acquisition,

Familiarity, Concreteness, and Imageability or ease with which an object could be

represented with a drawing (Juhasz, 2005). Although these ratings were not available

for all the English words we had selected, we report in Table 1 the information

available in the MRC psycholinguistic database4 (Wilson, 1988) for a random subset

of items in each condition. It approximates the characteristics of the whole set,

confirming that good multidimensional balance was reached between conditions,

hence that these parameters could not cause the variations observed in the results.

In the absence of a psycholinguistic lexical database for the Norwegian vocabulary,

the above psychological dimensions were impossible to control for the Norwegian

words. However, their formal homogeneity (length and frequency) could be computed

(Blair, Urland & Ma, 2002) and was high between conditions (Table 1). All materials

were cross-validated by a third party expert.

11
Table 1

Formal and Psychological descriptors of the English and Norwegian stimuli

N Lex
AoA Fam Concr Imag
Letters Frq

40/40/4 40/40/4 10/18/3 27/32/3 27/31/3 27/32/3

N words
0 0 5 7 7 7
222-
English stimulus word

10-
383
3-8 1790 424-620 311-614 384-634
150-
Min-Max 3-7 1-1430 441-644 482-635 412-630
406
3-7 20- 401-626 323-623 397-632
166-
1323
511
4.33/1. 324/38
01 3
282/52 552/52 500/92 537/67
4.75/0. 385/38
Mean/SD 268/67 548/53 589/32 574/43
99 0
304/64 542/48 519/83 543/60
5.03/1. 347/31
15 3
40/40/4 40/40/4
N words
0 0
Norwegian stimulus word

0.004-
6.3
3-10
0.032-
Min-Max 3-7
8.4
3-9
0.009-
NA
6.3
0.87/1.
5
5/1.5
1.23/1.
Mean/SD 4.75/1
8
5.1/1.6
0.77/1.
3
Note. N Letters: The amount of letters composing a word, or word length. Lex Frq:

Lexical frequency. For the English words, Thorndike-Lorge written frequency index

provided by the MRC Psycholinguistics Database. For the Norwegian words, index

computed by the author with the Google search engine (amount of pages written in

Norwegian, located in Norway and containing the target Norwegian word at least

once, expressed in millions of pages). AoA: Age of Acquisition. Fam: Familiarity.

Concr: Concreteness. Imag: Imageability. AoA, Fam, Concr and Imag are ratings

12
between 100 and 700, corresponding to a scale of 1 to 7. N words: Amount of words

per category. Values in each cell are given in this order: DC, TC and NC.

Experimental plan
Four lists, each containing one fourth of the 240 items, were created such that in

each list, half of the items were transcoding correspondences, while the remaining

half were not. Each participant was tested with two complementary lists, in order to

provide data on each of the 120 English words. Additionally, a training set containing

ten items was created.

Stimulation procedure
Items were selected randomly from each list and presented automatically, delivered

visually at 800 by 600 screen resolution and normal pixel density (96 DPI) on a high

quality 21” cathode ray tube screen (DELL P1130 Trinitron). The screen was

refreshed at 100 Hz, stimulus delivery being synchronized with screen refreshment

for improving timing precision of the stimulating procedure. The distance between the

surface of the screen and the subject’s eyes was 55 cm. Words were written with the

fixed-width font Courier New, in size 24 (9 mm height per letter, i.e. 0.94° visual

angle). The two words composing a stimulus were separated by an empty space

which varied between 9 and 18 characters depending on the length of the words.

This visual layout allowed us to keep the total item length constant (24 characters,

space included), as shown in Figure 2.

13
Figure 2. Items’ visual layout (character boxes were not visible in the stimulus)

Furthermore, each item’s length was made large enough to make it impossible

to read a word while gazing at the other one, whatever the word length. Therefore,

participants were obliged to make a saccade (a fast movement of the eyes) to read

each word, an essential constraint for analyzing the data: saccadic movements

produce a specific artefact in the EEG which could be used for clear time

synchronization with the onset of processing of the Norwegian word.

The stimulation sequence started with the English word (WL 2 - see Figure 3)

appearing randomly on the left or right half of the screen for 400 ms. Displaying WL 2

for this short duration served as a signal attracting the gaze of the subject to ensure

that the English word was always read first. After 400 ms, the Norwegian word (WL 1)

was displayed on the other half of the screen, while the English word remained on

the screen at its position. The two words were then displayed together until the

response key was pressed or for a maximum duration of two seconds. Two additional

seconds separated the end of an item’s presentation from the beginning of the next

(termed “inter-stimulus interval”). Every 20 items, a resting time was offered to the

participant. He was free to use it or not, for as long as he needed.

14
Figure 3. Summary of the experimental situation during the stimulation sequence on

one item

Task
Participants performed a “go, no-go” transcoding validation task, consisting of

checking if the English and the Norwegian words (DCs, TCs, NCs) presented in the

same item were transcoding correspondences, and pressing a button on a button

box if the Norwegian word was the correct transcoding of the English word or doing

nothing otherwise. Placed under performance pressure both for speed and accuracy,

subjects had to provide a decision as fast as possible and were encouraged to avoid

mistakes (see the Experimental Session section below for more details about the

experimental protocol).

Recording
Three computers were used. The first computer, piloted with a script written for E-

Prime (2008b),5 presented the stimulus and measured participant’s decisions (i.e.

15
key pressed and reaction time measured from the appearance of the Norwegian

word - see Figure 3). The second computer recorded the EEG continuously at 512

Hz with the ASA software (2008a), through a 64 channels shielded cap.6 The third

computer recorded eye-movements from the appearance of the English word and

until the subject had given his answer or until 2 seconds had elapsed. The eye-

tracker (EyeLink 1000, SR Research Ltd., Canada, used in remote mode with a 500

Hz monocular sampling rate) was piloted directly from the first computer by additional

instructions included in the E-Prime script. Computers 2 and 3 were time-

synchronized with Computer 1 through communication protocols (respectively

Parallel port and Etherlink), EEG and EYE data being time-stamped with triggers

generated by E-Prime for the various stimulation events and for the participant’s

answer. The time accuracy of the equipment, programming and communication

protocols were set and tested in a series of pre-tests.

Population
We were advised by the European Union Officers following EYE-to-IT to use

bilinguals only at this early stage of research development. Translators are a rare and

expensive population, whose use was not scientifically justified for this study. Indeed,

any bilingual could transcode isolated words as well as any expert translator. 52

participants were tested (52 per cent female), aged 27 on average (min 18, max 65,

SD 9). Norwegian native speakers who were also fluent bilinguals in English were

invited to undergo a two step selection procedure. A questionnaire first controlled

their linguistic background and linguistic competence, checking that candidates

learned English in early childhood (average age of commencement for learning

English was 6.5 years, min. 0 for real bilinguals, max. 10, SD 3.3). Those with an

adequate profile were given the online language assessment test “Dialang” (Freie

16
Universität Berlin et al., 2003), evaluating their proficiency in English. Participants

with the highest reading level and a vocabulary level score above 900 out of 1,000

were selected. After signing a consent form, candidates were included in the pool of

experimental subjects. None of the participants had cognitive impairment or

neurological disorders. Those with corrected vision were accepted provided that they

wore glasses or contact lenses during the experiment. Participants were paid 500

Norwegian Kroner.

Experimental session
Experimental sessions were individual and lasted 2 hours, including the testing on all

experiments. The participant was prepared (installation and plugging in of the electro-

cap), instructed on how to behave with the equipment (be as still as possible, blink

systematically between items, be extremely focused on the screen, use resting

breaks to relax the eyes and to move if necessary), and instructed about the task to

perform (read the stimulus, decide if the Norwegian word is the correct translation of

the English word, and press the button of the button box if the answer was yes, do

nothing otherwise. Perform the task as fast as possible and, at the same time, avoid

making mistakes). Instruments were then adjusted and tested (lowest possible

electric resistance for all electrodes, calibration of the eye-tracker). Before starting to

train on a set of training items, the subject was asked to summarize what he had

understood and remembered, and if needed, clarifications were given by the

experimenter. The subject was then trained until his performance reached a minimum

threshold of average speed (1200 ms) and accuracy (80 per cent). The testing part

which followed was extremely brief (five minutes long) because of its high intensity.

The experiment was run along with two other experiments in the same session, and

17
took place at the ERP Laboratory of the National Centre for Epilepsy, Rikshospitalet

University Hospital, Oslo, Norway, in June 2008.

Data processing
EEG: Time-Frequency analysis (Harmony et al., 2009; Hoechstetter et al., 2004;

Koenig, Hubl & Mueller, 2002) used the data from 44 participants only, the EEG

quality from eight participants being poor. Signal processing and analysis was done

with the software BESA (2009). Time-Frequency analysis considered EEG segments

of 575 ms (-100 to 475) around the first horizontal eye movement after the

appearance of the Norwegian word (beginning of the mental matching between L 2

and L1). Longer segments could not be used (appearance of the first answers). For

each Stimulus Type condition, the analysis compared power differences existing in

the EEG between the Matching condition and its corresponding Mismatching

condition, in 2 Hz frequency bands between 4 and 50 Hz, on each 25 ms time

interval between -100 and +475 ms. Time-frequency matrices of p-values7 were

obtained, one p-value per 2 Hz by 25 ms sample (Time-Frequency sample), one

matrix per channel, repeated for each one of the three Stimulus Type conditions. The

distribution of p-values across conditions, frequency bands, time intervals, and

cranial locations was analysed with multilevel modelling (Clark, 1973; Hox, 2002;

Lachaud & Renaud, 2009; Rasbash et al., 2009).

EYE: Eye data from all subjects were used. Raw measurements on matching

items were isolated from those on mismatching items. Items with blinks or no eye

detection were eliminated. The overall quality, however, was very good, because

subjects were instructed to blink systematically between items and to pay great

attention to the stimulus, which naturally reduced their blinking rate during item

presentation. A script written by Dr. O. Spakov (EYE-to-IT, TAUCHI, University of

18
Tampere, Finland), analysed the raw data and extracted Total Fixation Duration and

Pupil Size Amplitude Variation. Outliers were filtered with Median ± 3 Median

Absolute Deviation applied by subject and by item in each condition. Missing values

were not replaced. Three linear multilevel models, one per dependent variable,

crossed subjects and items as random variables and included Stimulus Type as a

fixed variable.

KEY: Data from only 41 subjects were used, 11 subjects having performed the

task incorrectly during part of the experiment. The procedure described above for eye

movements was followed for filtering outliers and statistical analysis. Because Error

Probability was binary, a logit binomial multilevel model was used. The model’s

structure is identical to the ones previously described.

For EEG, EYE and KEY, the test statistic was a large sample Chi squared test.

Results are graphed in Figures 4 to 8 with 95 per cent confidence interval bars.

19
Results
EEG

Figure 4. Time-Frequency Power Difference (in Time-Frequency unit) as a function of

Stimulus Type

20
 Time-Frequency Power Difference was smaller by 0.311 Time-Frequency units

with TCs than with DCs (2(1, N = 2736) = 36.8, p < .0001), and smaller by 0.238 Time-

Frequency units than with NCs (2(1, N = 2736) = 21.5, p < .0001). The difference

between DCs and NCs was marginal (p < .16).

EYE

Figure 5. Total Fixation Duration (in ms) as a function of Stimulus Type

21
 Total Fixation Duration with DCs was 368 ms longer than with TCs (2(1, N = 2827)

= 50.2, p < .0001), and 289 ms longer than with NCs (2(1, N = 2827) = 30.9, p < .0001).

The difference between TCs and NCs was marginal (p < .13).

Figure 6. Pupil Size Amplitude Variation (in arbitrary diameter unit) as a function of

Stimulus Type

22
 Pupil Size Amplitude Variation with DCs was 5.3 diameter units bigger than

with TCs (2(1, N = 2892) = 10.5, p < .0013) and 3.3 diameter units bigger than with NCs

(2(1, N = 2892) = 4, p < .047). The difference between TCs and NCs was not significant

(p < .22).

KEY

Figure 7. Reaction Time (in ms) as a function of Stimulus Type

Reaction times with DCs were 189 ms longer than with TCs (2(1, N = 2186) = 99,

p < .0001), and 127 ms longer than with NCs (2(1, N = 2186) = 44, p < .0001).

Additionally, reaction times were 62 ms longer with NCs than with TCs (2(1, N = 2186) =

11, p < .001).

23
Figure 8. Error Probability (in per cent) as a function of Stimulus Type

Error Probability with DCs was 15 per cent bigger than with TCs (2(1, N = 2460) =

24, p < .0001), and 14 per cent bigger than with NCs (2(1, N = 2460) = 20, p < .0001).

The difference between TCs and NCs was not significant (p < .66).

24
Discussion
Result patterns across dependent variables give a clear, indisputable picture of the

greater difficulty participants had transcoding DCs (positive Time-Frequency Power

Difference, longest Total Fixation Duration, biggest Pupil Size Amplitude Variation,

longest Reaction Time and biggest Error Probability). The fact that TCs were easier

to process is also indisputable (biggest negative Time-Frequency Power Difference,

shortest Total Fixation Duration, smallest Pupil Size Amplitude Variation, shortest

Reaction Time and smallest Error Probability). Although an intermediate pattern was

predicted for NC, analyses do not clearly confirm the hypothesis. Yes, the dependent

variable’s average for NCs is always between DCs and TCs, but the difference

between the three conditions is only significant for reaction times. With other

dependent variables, NCs are either significantly different from TCs and not from

DCs (Time-Frequency Power Difference), or from DCs and not from TCs (Total

Fixation Duration, Pupil Size Amplitude Variation and Error Probability).

If the study had been based on one dependent variable only, the chances are

high that an incorrect conclusion would have been given, e.g.: “NCs are as difficult to

transcode as DCs”, or “NCs are as easy to process as TCs”. It is, however, likely that

NCs are more difficult to process than TCs, but easier to transcode than DCs.

Because each dependent variable measures a different aspect of the mind’s activity,

it is not surprising that varying patterns of results are obtained, hence the

fundamental importance of choosing dependent variables adequately targeting the

mental process that needs to be studied.

This study reveals that the processing difficulty (Reaction Time) is the highest

for DCs, the lowest for TCs, and medium for NCs. The verification effort (Total

Fixation Duration) and reading attention demand (Pupil Size Amplitude Variation) is

the highest for DCs and the lowest for TCs. The verification effort is marginally bigger

25
with NCs than with TCs, but attention demand is not statistically different. Participants

were the most confused with DCs and less but equally confused with TCs and NCs.

The overall cognitive load is strongly reduced between the TC Mismatching and

Matching conditions, and slightly reduced with NCs, indicating that less computation

is required to make a decision in these matching conditions than in the corresponding

mismatching conditions. The overall cognitive load is slightly increased with DCs,

indicating that more computation was needed to make a decision in the matching

condition than in the mismatching condition.

The Conceptual Mediation Hierarchical Model presented in Figure 1 is

supported by this set of results. Because the lexical form is identical between L 2 and

L1 for TCs, verification effort, attention demand, difficulty and confusion are the

lowest and cognitive load is reduced in the matching condition compared to the

mismatching condition. With NCs, the meaning of the two words is identical and

serves as a link between L2 and L1 lexical forms. Thanks to this semantic link, the

overall cognitive load is slightly reduced in the matching condition compared with the

mismatching condition. Verification effort is greater with NCs than with TCs because

two lexical forms are now being compared and mentally transcoded. Consequently,

attention demand, confusion level and the decision-making load are also higher. DC

words have the same lexical form in L2 and L1, but a different meaning in L2 and L1.

DCs therefore induce the highest confusion and difficulty, as well as hesitation in the

decision making process. Thinking about possible meanings, bilinguals will eventually

follow one of the four paths described in the introduction. Due to the additional

computation that was not necessary in the NC condition, it is logical to find an overall

cognitive load increase in the matching condition compared to the mismatching

condition.

26
Conclusion
This study has demonstrated the possibility and the added value of simultaneously

using multiple techniques to measure the cognitive load involved in transcoding TCs,

NCs and DCs. The methodology has allowed us to confirm the existence of a transfer

difficulty hierarchy that translators may face at the word level: DCs are the most

difficult to process, NCs represent an intermediate cognitive load, and TCs are the

easiest. The results allow us to pave the way for a Translation Priming system that

might help translators retrieve forms and meanings from long-term memory. For

example, when processing DCs translators might benefit from prompts related both

semantically and lexically to the target. On the other hand, the processing of NCs

might only require prompts at the lexical level. Finally, when processing TCs

translators might not benefit from prompting as they would not be facing any

transcoding challenge. This information was essential for grounding the next steps of

the study: testing the benefits of prompting on transcoding (Lachaud, Fougner

Rydning & Larsson, Submitted) and translation (future research).

27
1
Development of Human-Computer Monitoring and Feedback Systems for the Purposes of Studying
th
Cognition and Translation, 2006/01-2009/04, 6 Framework Program, Information Society
Technologies (Contract 517590).
2
According to this type of psychological model, two languages share the same semantic system of
representation in the bilingual mind, but each language is stored as a distinct system of formal
representations. Both languages are therefore linked together through the common semantic system -
Illes, J., Francis, W. S., Desmond, J. E., Gabrieli, J. D. E., Glover, G. H., Poldrack, R., Lee, C. J. &
Wagner, A. D. (1999). Convergent Cortical Representation of Semantic Processing in Bilinguals. Brain
and Language, 70(3), 347-363.
3
Pupil size provided by the EyeLink 1000 is given as integer numbers in an arbitrary unit, with a noise
level of 0.2 per cent of the diameter.
4
The MRC Psycholinguistic database: Cf. http://www.psych.rl.ac.uk/MRC_Psych_Db.html (Last
accessed 06/01/10).
5
E-Prime is software specifically developed for psychological research. It allows the presentation of
visual, auditory, or audio-visual stimuli to human subjects, and records their behavior in automated
experimental sessions.
6 TM
WaveGuard cap (ANT) with a 64 channels 10-20 international layout, using 64 sintered Ag/AgCl
electrodes, Hirose HD connectors.
7
Statistical index giving the probability that the null hypothesis is true. The scientific standard
threshold is p < .05. It means that the probability to observe a given pattern in the data due to chance
is below 5 per cent.

References

(2008a). Advanced Signal Analysis. Enschede, Netherlands: Advanced Neuro

Technology Software BV (ANT).

(2008b). E-Prime. Pittsburgh, PA, USA: Psychology Software Tools, Inc.

(2009). Brain Electrical Source Analysis. Gräfelfing, Germany: MEGIS Software

GmbH.

Blair, I. V., Urland, G. R. & Ma, J. E. (2002), 'Using Internet search engines to

estimate word frequency'. Behavior Research Methods, Instruments &

Computers, 2 (34), 286-290.

Bölte, J. & Coenen, E. (2002), 'Is Phonological Information Mapped onto Semantic

Information in a One-to-One Manner?' Brain and Language, 81 (1-3), 384-397.

Cavero, L., Concejero, P., Gili, J., Yuichiro Anzai, K. O. & Hirohiko, M. (1995), 'Help

and prompting in broad band multimedia services'. Advances in Human

Factors/Ergonomics, pp. 225-230. Elsevier.

28
Chamizo Domínguez, P. J. & Nerlich, B. (2002), 'False friends: their origin and

semantics in some selected languages'. Journal of Pragmatics, 34 (12), 1833-

1849.

Clark, H. H. (1973), 'The language-as-fixed-effect fallacy: a critique of language

statistics in psychological research'. Journal of Verbal Learning and Verbal

Behavior, 12, 335-359.

Duyck, W. & Warlop, N. (2009), 'Translation Priming Between the Native Language

and a Second Language: New Evidence From Dutch-French Bilinguals'.

Experimental Psychology, 56 (3), 173-179.

Ferreira, A. & Atkinson, J. (2009), 'Designing a feedback component of an intelligent

tutoring system for foreign language'. Knowledge-Based Systems, 22 (7), 496-

501.

Finkbeiner, M., Forster, K., Nicol, J. & Nakamura, K. (2004), 'The role of polysemy in

masked semantic and translation priming'. Journal of memory and language,

51 (1), 1-22.

Freie Universität Berlin, G., Jyväskylän yliopisto, F., Lancaster University, U., CITO

groep, N., Karl-Franzens-Universität Graz, A., Vrije Universiteit Brussel, B.,

Handelshøjskolen i Århus, D., Århus Kommunes Sprogcenter, D., Generalitat

de Catalunya, E., Aristotelio PanepistimioThessalonikis, G., ITÉ Dublin, I.,

Háskóli Íslands, I., INVaISI, I., Universitetet i Bergen, N., Universidade de

Aveiro, P., Universidade de Coimbra, P., Göteborgs Universitet, S. & Lunds

Universitet, S. (2003). DIALANG. European project for the development of

diagnostic language tests in 14 European languages. French, R. M. &

Jacquet, M. (2004), 'Understanding bilingual memory: models and data'.

Trends in Cognitive Sciences, 8 (2), 87-93.

29
Frenck, C. & Pynte, J. (1987), 'Semantic representation and surface forms: A look at

across-language priming in bilinguals'. Journal of Psycholinguistic Research,

16 (4), 383-396.

Group, C. P. R. (1986), Parallel distributed processing: explorations in the

microstructure of cognition. Cambridge, MA, USA: MIT Press.

Harmony, T., Alba, A., Marroquín, J. L. & González-Frankenberger, B. (2009), 'Time-

frequency-topographic analysis of induced power and synchrony of EEG

signals during a Go/No-Go task'. International Journal of Psychophysiology,

71 (1), 9-16.

Hoechstetter, K., Bornfleth, H., Weckesser, D., Ille, N., Berg, P. & Scherg, M. (2004),

'BESA Source Coherence: A New Method to Study Cortical Oscillatory

Coupling'. Brain Topography, 16 (4), 233-238.

Hox, J. (2002), Multilevel analysis: techniques and applications. London: Lawrence

Erlbaum Associates.

Illes, J., Francis, W. S., Desmond, J. E., Gabrieli, J. D. E., Glover, G. H., Poldrack,

R., Lee, C. J. & Wagner, A. D. (1999), 'Convergent Cortical Representation of

Semantic Processing in Bilinguals'. Brain and Language, 70 (3), 347-363.

Juhasz, B. J. (2005), 'Age-of-Acquisition Effects in Word and Picture Identification'.

Psychological Bulletin, 131 (5), 684-712.

Koenig, T., Hubl, D. & Mueller, T. J. (2002), 'Decomposing the EEG in time, space

and frequency: a formal model, existing methods, and new proposals'.

International Congress Series, 1232, 317-321.

Lachaud, C. M., Fougner Rydning, A. & Larsson, P. G. (Submitted). 'Towards Online

Automatic Systems for Improving Human Text Translation: Transcoding

30
 Performance and Cognitive Load Variations after Prompting'. International

Journal of Human-Computer Studies.

Lachaud, C. M. & Renaud, O. (2009), 'A Tutorial for Analyzing Human Reaction

Times: How to Filter Data, to Manage Missing Values, and to Chose a

Statistical Model?'. Applied Psycholinguistics, accepted.

Lederer, M. (2006). Défense et illustration de la Théorie Interprétative de la

Traduction (p. 132). In F. Israël & M. Lederer (Eds.), La Théorie Interprétative

de la Traduction: Genèse et Développement. Paris: Miraud.

McClelland, J. L. & Elman, J. L. (1986), 'The TRACE model of speech perception'.

Cognitive Psychology, 18 (1), 1-86.

Niedermeyer, E. & Da Silva, F. L. (2005). Electroencephalography Basic Principles,

Clinical Applications, and Related Fields. Philadelphia, PA, USA: Lippincott

Williams & Wilkins.

Rasbash, J., Charlton, C., Browne, W. J., Healy, M. & Cameron, B. (2009), MLwiN.

Bristol, UK: Centre for Multilevel Modelling, University of Bristol.

Schwanenflugel, P. J. & Rey, M. (1986), 'Interlingual semantic facilitation: Evidence

for a common representational system in the bilingual lexicon'. Journal of

memory and language, 25 (5), 605-618.

Wilson, M. D. (1988), 'The MRC Psycholinguistic Database: Machine Readable

Dictionary, Version 2'. Behavioural Research Methods, Instruments and

Computers, 20, 6-11.

31

View publication stats