This article was downloaded by: [b-on: Biblioteca do conhecimento online UTAD]

On: 11 February 2013, At: 09:56

Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer
House, 37-41 Mortimer Street, London W1T 3JH, UK

Cognitive Neuropsychology
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/pcgn20

NORMAL AND IMPAIRED SPELLING IN A

CONNECTIONIST DUAL-ROUTE ARCHITECTURE
a b
George Houghton & Marco Zorzi
a
University of Wales, Bangor, UK
b
Università di Padova, Italy
Version of record first published: 09 Sep 2010.

To cite this article: George Houghton & Marco Zorzi (2003): NORMAL AND IMPAIRED SPELLING IN A CONNECTIONIST DUAL-
ROUTE ARCHITECTURE, Cognitive Neuropsychology, 20:2, 115-162

To link to this article: http://dx.doi.org/10.1080/02643290242000871

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-conditions

This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to
anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents
will be complete or accurate or up to date. The accuracy of any instructions, formulae, and drug doses
should be independently verified with primary sources. The publisher shall not be liable for any loss, actions,
claims, proceedings, demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
 Q1030–CN3301 / Mar 4, 03 (Tue)/ [48 pages, 9 tables, 12 figures, 8 footnotes, 3 Appendices] – S endings, c.f. [no
comma]. DISK Edited - Maths Equations & Phonetics to set
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2), 115–162

NORMAL AND IMPAIRED SPELLING IN A

CONNECTIONIST DUAL-ROUTE ARCHITECTURE

George Houghton
University of Wales, Bangor, UK
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

Marco Zorzi
Università di Padova, Italy

This paper presents a dual-route connectionist model of spelling, in which one route maps directly from
sound to spelling (phonemes to graphemes), while in the other route the mapping is mediated by a fur-
ther level of representation. The direct route is implemented as a two-layer associative network, with
syllabically structured phonemic (input) and graphemic (output) representations, which comes to
behave as a productive sound-to-spelling conversion mechanism through the exposure to a corpus of
monosyllabic words. The mediated route is modelled as a frequency-sensitive lexical pathway. Nodes
representing more frequent words become activated more rapidly than those of lower-frequency words.
Access to both routes occurs in parallel, and the final spelling is determined by the combined output of
both routes. We show that the model accounts for a wide range of data from normal spellers (including
nonword spelling, the variability in vowel spelling and the effect of surrounding phonological context,
frequency effect and its interaction with spelling regularity). We also investigate the effect of a selective
lesion to the lexical route in which the ceiling of lexical activation is lowered. This manipulation pro-
duces a model with surface dysgraphic characteristics, which is tested against data from two impaired
subjects. As well as simulating the classic surface dysgraphic profile, including a frequency by regularity
interaction, the model exhibits a phenomenon that has only recently been reported, and which provides
strong evidence for the idea that multiple routes are active in parallel, and combine to produce the final
spelling.

INTRODUCTION quence of learning to read and write in an alpha-

In literate societies, a considerable part of our expe-
rience of language involves its written form. For
children, learning to read and write is the most
important goal of their early school years, and pro-
vides the indispensable foundation for all later for-
mal education. Literacy may have effects on other
language skills; for instance, there is evidence that
awareness of individual phonemes arises as a conse-
betic script (Goswami & Bryant, 1990; Morais,
Cary, Alegria, & Bertelson, 1979). It is also likely
that the acquisition of literacy, and even the nature
and regularity of the writing system learned, has
implications for brain development. For instance,
in a recent neuroimaging study comparing reading
in English and Italian, Paulesu et al. (2000) report
differences in the distribution of brain activation as
a function of the “depth” of the orthography.

Requests for reprints should be addressed to George Houghton, School of Psychology, University of Wales, Bangor, Bangor,
Gwynedd, LL57 2DG, UK (Tel: (0)1248 382692; Fax: (0) 1248 382599; Email: g.houghton@bangor.ac.uk).
We are grateful to Michael Cortese, David Plaut, Brenda Rapp, and Marie Josephe Tainturier for their helpful comments on an
earlier version of this article.

 2003 Psychology Press Ltd 115

http://www.tandf.co.uk/journals/pp/02643294.html DOI:10.1080/02643290242000871
 HOUGHTON AND ZORZI
It is no surprise, then, that educational, cogni-
tive, and neuro-psychologists have a long-standing
interest in the processing of written language.
Although more experimental work has been done
on reading than on writing, the production of writ-
ten language, in particular spelling, has not been
neglected (Brown & Ellis, 1994; Frith, 1980).
However, in the last decade or so the empirical
work on reading has been accompanied by a vigor-
ous programme of theoretical work, involving com-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
putational modelling of many aspects of normal
and impaired reading by a number of different
groups (e.g., Brown, 1987; Bullinaria, 1997;
Coltheart & Rastle, 1994; Coltheart, Rastle, Perry,
Langdon, & Ziegler, 2001; Grainger & Jacobs,
1996; Harm & Seidenberg, 1999; Hinton &
Shallice, 1991; McClelland & Rumelhart, 1981;
Norris, 1994; Plaut, McClelland, Seidenberg, &
Patterson, 1996; Plaut & Shallice, 1993;
Seidenberg & McClelland, 1989; Sejnowski &
Rosenberg, 1987; Zorzi, 2000; Zorzi, Houghton &
Butterworth, 1998a, 1998b). Research on spelling
has not so far given rise to a theoretical literature of
comparable breadth and depth (see the General
Discussion section for review). In this paper, we
present the first model of spelling with a scope
comparable to that achieved in some of the models
of reading.
Spelling and the dual-route framework
In alphabetic scripts, the written symbols are
intended to stand for words by virtue of represent-
ing their pronunciation; in particular, written
graphemes (single letters, or letter combinations
such as SH in English, or LL in Welsh) stand for
spoken phonemes. This way of representing spoken
language was a significant cultural invention, and
evolved from earlier logographic scripts in which
individual symbols had semantic rather than pho-
nological significance (Gelb, 1952). The advantage
of an alphabet lies in its economy and flexibility. No
1
Phonemic symbol key (British English pronunciation): Vowels: /A/ as in cAR, /&/ as in cAt, /e/ as in bEt, /I/ as in hIt, /o/ as in
hOt, /V/ as in hUt, /i / as in bEAt, /u/ as in bOOt, /U/ as in pUt, /O/ as in dOOR, /@U/ as in lOAf, /aI/ as in fIle, /eI/ as in wAIt, /oI/
as in sOIl, /3/ as in bURn, /I@/ as in chEER. Consonants: most have standard values, e.g., /d/ as in Door. Also /dZ/ as in Jet, /S/ as in
SHed, /tS/ as in CHin, /9/ as in siNG, /T/ as in THin, /D/ as in THat.
116 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
language has more than a few dozen phonemes,
compared to thousands of words and morphemes.
Thus an alphabetic script allows an entire language
to be written with relatively few symbols, and the
writing of new words (for instance, foreign loan
words and place names) does not require the inven-
tion of new symbols.
For the psychologist, one of the most striking
features of alphabetic writing is that in principle it
provides more than one method by which a word
may be spelled (or read). Take the English word
“bat.” Intuitively, one could write it correctly
either by learning it as a sequence of three letters,
or by translating its constituent phonemes /b&t/1
into letters, /b/ = B, /&/ = A, /t/ = T. In a com-
pletely regular, consistent alphabetic script, the
second method would always generate the canoni-
cal spelling, and many modern orthographies
come very close to this ideal (German, Spanish,
Welsh, Italian, etc.). Others, however, do not, and
considerable attention has been paid to the “quasi-
regularity” problem—in English, for instance,
sound-to-spelling relationships are basically regu-
lar but with many inconsistencies and specific
exception words.
The relationship between the pronunciation and
the spelling of a word can be used to classify it with
respect to how regular and consistent its spelling is
relative to the major sound–spelling regularities of
the language. A broad but useful classification dis-
tinguishes between regular-consistent (RC), regu-
lar-inconsistent (RI), and irregular or exception
(EXC) words. Words of the first two categories are
regularly spelled in the sense that the sounds of the
word are represented by graphemes that represent
the same sounds in many other words. In the case of
RC words, the sound-to-spelling relationship is
also consistent, in that the sounds in question are
always (or almost always) spelled the same way. For
instance, words such as plank, lift, shop, thing, etc.
are both regular and consistent. Each phoneme is
represented by the grapheme that always represents

that sound in the context in which it occurs. So all
English words that begin with the dental fricative
/T/ begin with the letters TH; (virtually) all mono-
syllabic words containing the short vowel /I/ are
spelled with an I. By contrast, regular-inconsistent
words contain sounds that may be spelled in more
than one way. The major locus of inconsistency in
English spelling is in the vowel (see Kessler &
Treiman, 2001, for a quantitative analysis). For
instance, while words such as feet, corn, paid, etc.
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
show a “regular” spelling pattern, the vowels in
them are often represented by other graphemes. So
the vowel /i/ of feet is often represented by the
grapheme EA (e.g., treat, feat). In British English,
the long /O/ of corn could be represented by the
grapheme AW (e.g., yawn, lawn, brawn), and so
on. Exception, or irregular, words contain at least
one very low-frequency phoneme–grapheme
relationship, e.g., /eI/ → EA in great, /V/ → 0 in
front, /S/ → CH in chef. It is important to note here
that the classification of words in this way is not
independent of the direction of the sound–spelling
relationship one is considering. Thus some words
that are regular and consistent from the point of
view of reading (e.g., green could only be pro-
nounced /grin/), are inconsistent with respect to
spelling (/grin/ could be spelled grean, by analogy
with mean, lean, bean, etc., or possibly even grene, as
in scene; see Ziegler, Stone, & Jacobs, 1997).
The standard way of assessing the psychological
“strength” of a particular sound–spelling relation-
ship is to test its productivity in reading or spelling
phonologically legal nonwords (or pseudowords).
Literate adults can easily read and write such words
and generally do so in a way that conforms to the
dominant regularities of the language. To take an
example from reading, the vowel letter I, when fol-
lowed by a group of consonant letters, is virtually
always pronounced as in hint, pick, swim, grist, etc.
However, in the exception word pint it corresponds
to a diphthong. But the pseudoword WINT is most
likely to be pronounced to rhyme with lint rather
than with pint. Where there is more inconsistency
in a mapping, this often manifests itself as greater
variability in the responses to pseudowords. For
instance, in a study on nonword spelling by Barry
and Seymour (1988), all participants spelled the
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
vowel in the pseudoword /pIv/ with an I. In con-
trast, the majority of participants spelled the
pseudoword /teIn/ in one of two different ways,
tane (63%) or tain (37%—the percentages are
derived from raw scores reported in Barry &
Seymour, 1988, Appendices 1 and 2).
The idea that the sound-to-spelling regularities
of an alphabetic script may be separately repre-
sented from knowledge of the spelling of individual
word has a long history (see, e.g., Baron &
Strawson, 1976; Coltheart, 1978, 1985; Forster &
Chambers, 1973; Morton, 1980), and has given rise
to the “dual-route” framework for both reading and
spelling. This is illustrated in Figure 1 for the case
of spelling.
One route of this framework, which we will refer
to as the lexical route, is postulated to be memory
based. Through extensive experience, literate adults
come to memorise the spelling of individual words,
and these are stored in the orthographic output lexi-
con (OOL). How this is done, and the nature of the
lexical representations formed, is not specified by the
framework, and may differ extensively between
models that conform to it (see Shallice, Glasspool, &
Houghton, 1995, for a comparison between two lex-
ical spelling models). These lexical representations
can be accessed in various ways, for instance from the
sound of a familiar word, or from its meaning (see
Barry, 1994, for discussion of a number of possible
Phonological
input Semantics
Phonological
input lexicon
Sound–spelling Orthographic
conversion output lexicon
Graphemic
output buffer
Handwriting Typing
Figure 1. A typical “dual-route” model for spelling. Dotted arrows
show the lexical route, bold arrows the sound–spelling conversion
route. A direct input from semantics is also shown.
117
 HOUGHTON AND ZORZI
routes to the OOL). The second route deals with the
sublexical sound–spelling conversion process, and
we will generally refer to this as the phonological route
(other terms used include the assembly route, sound-
to-spelling route, and phoneme–grapheme conversion).
Again, the dual-route framework itself does not
specify how this route is formed or how it operates,
and many possibilities are compatible with the
framework. For instance, the analogous route for
reading (spelling-to-sound conversion) is modelled
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
by Coltheart and Rastle (1994) as a serial-process-
ing, symbolic production system, and by Zorzi et al.
(1998a) as a parallel-processing, associative network
(see Zorzi, 2000, for further discussion of serial vs.
parallel processing in phonological assembly). A fur-
ther possible source of variation amongst models
concerns the nature of the interactions between the
two routes, and control over when each route is to be
used.
Despite the many important differences
amongst possible dual-route models, there is a cen-
tral point of agreement regarding the division of
labour within the system. The lexical route may be
used to spell any learned word, and is crucial at least
for the correct spelling of exception words. The
phonological route translates the constituent
sounds of words into letters on the basis of
sublexical sound-to-spelling regularities; hence
exception words, by definition, will be regularised
(misspelled) by this route. The phonological route
is at least used to spell novel, or poorly learned,
words. Its contribution to the spelling of known
words is not defined by the dual-route framework,
and depends on the particular theory. However, it is
generally supposed that any plausible mechanism
will be capable of generating the correct spelling of
at least the regular-consistent words (in languages
with very regular orthographies, this set will include
most of the words in the language).
Strong support for the basic assumptions of the
dual-route framework has been derived from stud-
ies of subjects with spelling impairments, and mod-
elling data from such studies is an important aim of
this paper. Before describing the model, we briefly
review data from neuropsychological studies sup-
portive of the basic functional division discussed
here.
118 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
Surface dysgraphia
The cognitive operations involved in the process of
oral and written spelling have been investigated in a
number of neuropsychological studies (Denes,
Cipolotti, & Zorzi, 1999; Shallice, 1988, for
reviews). Similarly to the case of reading disorders,
the study of acquired spelling disorders
(“dysgraphia”) provides strong evidence for a two-
process model of spelling. More specifically, the
observation of a neuropsychological double dissoci-
ation between the ability to spell known words (in
particular exception words) and the ability to spell
nonwords has been taken as evidence for the exis-
tence of functionally independent lexical and
sublexical spelling procedures.
First described by Beauvois and Derousné
(1981), surface dysgraphia is characterised by poor
spelling of words whose spellings cannot be unam-
biguously derived from their pronunciations. Sur-
face dysgraphic patients are impaired in spelling
words with exceptional sound–spelling correspon-
dences, whereas their ability to convert phonemes
into graphemes (i.e., nonword spelling) is well pre-
served. The syndrome was first termed lexical or
orthographic agraphia, to indicate that the disorder
originated from a deficit involving the lexical spell-
ing procedure. The complementary deficit, named
phonological agraphia, was described by Shallice
(1981) and is characterised by a dissociation
between a good ability to write words (both regular
and exception) and a poor performance in writing
nonwords to dictation. For example, the patient
described by Shallice wrote 94% of words correctly
but only 18% of nonwords.
A number of cases of surface dysgraphia have
been described (e.g., Baxter & Warrington, 1987;
Behrmann, 1987; Behrmann & Bub, 1992;
Coltheart & Funnell, 1987; De Partz, Seron, &
Van der Linden, 1992; Goodman & Caramazza,
1986; Goodman-Schulman & Caramazza, 1987;
Hatfield & Patterson, 1983; Rapcsak, Arthur, &
Rubens, 1988; Rapp, Epstein, & Tainturier, 2002;
Roeltgen & Heilman, 1984; Weekes & Coltheart,
1996). Most of the patients had stable focal lesions
resulting from stroke or brain injury, but the surface
dysgraphic pattern has also been observed in

patients with progressive disorders such as Alzhei-
mer’s dementia (e.g., Hughes, Graham, Patterson,
& Hodges, 1997; Lambert, Eustache, Viader,
Dany, Rioux, & Lechevalier, 1996; Platel et al.,
1993; Rapcsak, Arthur, Bliklen, & Rubens, 1989)
and semantic dementia (e.g., Graham, Patterson,
& Hodges, 1997, 2000). Surface dysgraphia also
has a developmental counterpart. A surface
dysgraphic pattern has been observed in some
developmental cases, usually associated with sur-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
face dyslexia (e.g., Castles & Coltheart, 1996;
Coltheart, Masterson, Byng, Prior, & Riddoch,
1983; Hanley, Hastie, & Kay, 1992).
Typical spelling errors in surface dysgraphia are
phonologically plausible, in the sense that reading
aloud the misspelled word will produce the correct
phonological form of the target. For instance,
the word VEIN might be misspelled as VANE,
but both spelt forms have the same pronunciation
(/veIn/). However, accuracy of spelling is also
affected by frequency: Surface dysgraphic patients
show a frequency-by-regularity interaction, that is,
their performance on exception words gets worse as
the frequency of the words decreases (Beauvois
& Derousné, 1981; Behrmann & Bub, 1992;
Coltheart & Funnell, 1987; De Partz et al., 1992;
Goodman & Caramazza, 1986).
A pure example of acquired surface dysgraphia is
MP, a patient studied by Behrmann and Bub
(1992). The case of MP is widely known as one of
the purest examples of surface dyslexia and has been
the subject of several papers (e.g., Behrmann &
Bub, 1992; Bub, Cancelliere, & Kertesz, 1985;
Patterson & Behrmann, 1997). Interestingly, the
surface dysgraphia showed by MP paralleled her
surface dyslexic pattern in a striking way. She
showed very good spelling of nonwords and regular
words, but was poor at spelling irregular words.
Furthermore, her performance with irregular words
declined as a function of their frequency and almost
all of her spelling errors were phonologically plausi-
ble (i.e., “regularisations”). We present a detailed
simulation of data from MP later in the paper.
Surface dysgraphia is usually associated with
lesions in the region of the left angular gyrus (e.g.,
Beauvois & Derousné, 1981; Behrmann, 1987;
Goodman & Caramazza, 1986; Goodman-
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
Schulman & Caramazza, 1987; Hatfield &
Patterson, 1983; Penniello et al., 1995; Roeltgen &
Heilman, 1984), although other regions (e.g., the
left precentral gyrus; Rapcsak et al., 1988) have
sometimes been implicated. However, the idea that
the functional role of the left angular gyrus is that of
the orthographic lexicon (whose impairment would
cause surface dysgraphia) is somewhat inconsistent
with the results of neuroimaging studies of reading,
which have related activation of the left angular
gyrus to semantic processing (e.g., Demonet et al.,
1992; Price, Moore, Humphreys, & Wise, 1997).
Furthermore, the regions associated to ortho-
graphic, “word form” processing in neuroimaging
studies of reading are more consistent with the
lesions that cause letter production problems in
spelling (i.e., a peripheral dysgraphia; Ellis, 1988).
High-resolution techniques (e.g., fMRI) are
needed to investigate whether a whole-word level
and a letter/grapheme level of orthographic repre-
sentation can be separated in an activation study,
because the associated regions are likely to be spa-
tially contiguous if not partially overlapping (see N.
L. Graham et al., 1997, for similar argument).
Spelling and semantics. Recent neuropsychological
studies have tackled the issue of the relationship
between semantic memory impairments and sur-
face dysgraphia. Graham et al. (2000) studied a
group of patients with semantic dementia and
found that their performance, compared to that of
control subjects, was disproportionately affected by
regularity and word frequency. Spelling of low-fre-
quency exception words was most impaired, and
the majority of errors were phonologically plausible
renderings of the target word. This pattern mirrors
the neuropsychological association between
semantic dementia and surface dyslexia that led
Patterson and Hodges (1992) to the hypothesis
that correct reading of low-frequency exception
words is dependent upon the integrity of semantic
representations (see Plaut et al., 1996, for a simula-
tion instantiating this theory). Indeed, many
patients presenting with semantic dementia are also
surface dyslexic (e.g., Funnell, 1996; Graham,
Hodges, & Patterson, 1994; Patterson & Hodges,
1992), but the corresponding dissociation (i.e.,
119
 HOUGHTON AND ZORZI
intact reading in the presence of semantic deficits)
has also been observed (e.g., Cipolotti &
Warrington, 1995; Lambon Ralph, Ellis, & Frank-
lin, 1995; Schwartz, Saffran, & Marin, 1980). With
regard to spelling, Hall and Riddoch (1997)
described the case of a patient (KW) who, in spite of
impaired auditory comprehension, showed a well-
preserved ability to write to dictation words (both
regular and irregular) that he could not understand.
In patients with dementia of the Alzheimer’s type
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
(DAT), who indeed present with semantic deficits,
the finding of surface dysgraphia is not universal,
and with disease progression nonphonologically
plausible errors increase (see Graham, 2000, for
review). Moreover, in DAT patients the effect of
spelling regularity is only slightly enhanced in com-
parison to control subjects (e.g., Glosser, Grugan,
& Friedman, 1999a; Glosser, Kohn, Sands,
Grugan, & Friedman, 1999b).
One possible explanation for the association
between semantic dementia and surface dyslexia/
dysgraphia is that it reflects pathological involve-
ment of brain regions that are closely related both
functionally and anatomically (also see Cipolotti &
Warrington, 1995). In other words, this specific
form of cortical degeneration would lead to seman-
tic impairments but also (and perhaps invariably) to
the disruption of lexical processing (both ortho-
graphic and phonological). This hypothesis would
seem to gain support from a recent study that com-
pared the reading performance of patients with
different types of dementia (Noble, Glosser, &
Grossman, 2000). Despite the presence of a seman-
tic impairment, patients with Alzheimer’s disease,
frontotemporal dementia, and progressive non-
fluent aphasia did not show a pattern of reading
difficulty consistent with surface dyslexia; only the
patients with semantic dementia showed the pre-
dicted pattern of reading impairment. However, it
is worth noting that the disruption of lexical ortho-
graphic and phonological processing in semantic
dementia appears to be tightly linked to the loss of
word meaning, as demonstrated by the correlation
across items between comprehension and exception
word reading (Funnell, 1996; Graham et al., 1994)
and spelling (Macoir & Bernier, 2002) that has
been observed in some patients. Funnell proposed
120 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
that in surface dyslexics a lexical response is pre-
ferred over a sublexical regularised response only
when vestiges of word meaning remain. Patient
EP, who presented with semantic dementia (and
surface dyslexia), showed knowledge of both lexi-
cally derived and sublexically derived pronunciation
in reading exception words (e.g., presented with
GLOVE she responded “/gl@Uv/ or /glVv/ ?”; see
Funnell, 1996). When all meaning of a given word
was lost, the patient selected a sublexical response,
even when she had also produced the correct lexical
response. In the same vein, Ward, Stott, and Parkin
(2000) documented the case of a patient with
semantic dementia (SA) who showed the ability to
use partial semantic knowledge to constrain his
reading and spelling.
Connectionist models of spelling: Single-
and dual-route architectures
As noted earlier, the experimental literature on nor-
mal and impaired reading has generated an expand-
ing literature in computational modelling, with
virtually all current models containing at least some
connectionist components. Although spelling has
not received the same degree of attention, a number
of connectionist models of spelling have been pro-
duced, using architectures similar to those of the
reading models of Sejnowski and Rosenberg (1987)
and Seidenberg and McClelland (1989). For
instance, Brown and Loosemore (1994), Bullinaria
(1994), and Olson and Caramazza (1994) all
describe models of sound–spelling conversion
using “single-route,” multilayer networks trained
with the backpropagation algorithm (Rumelhart,
Hinton, & Williams, 1986; these models are
described further in the General Discussion section
of this paper). Such networks contain layers of pro-
cessing units whose activation states represent the
inputs and outputs of the cognitive system under
study (phonemes and graphemes, respectively, in
the case of spelling). Mediating between these lev-
els is another layer of units (“hidden units”) that
receive activation via a set of learned weights from
the input units, and transmit their responses to the
output units via a second set of weights (see Figure
2b). This architecture for mapping between two

levels of representation has become quite common
in cognitive modelling, and we will refer to it as the
single-route, multilayer (or SR-ML) architecture.
As is well known, such multilayer networks have
greater representational power than two-layer net-
works in which the input and output domains are
directly connected. As such they can be considered
a “generalisation” of the feedforward perceptron of
Rosenblatt (1962; see Rumelhart et al., 1986, for
discussion). The provision of hidden units in the
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
intermediate layer permits the learning (in princi-
ple) of arbitrary nonlinear mappings, including the
item-by-item “rote” learning of the stimuli in the
training set. If the representations in the hidden
layer are distributed to some extent, then it is possi-
ble for such a network to learn to respond well to
exceptional input–output pairs, while at the same
time extracting the kind of statistically reliable rela-
tionships that permit a good generalisation to novel
stimuli. However, problems for such architectures
have arisen in the domain of written-language pro-
cessing due to the dissociations discussed above
between the surface and phonological forms of dys-
lexia and dysgraphia. Artificial lesioning of SR-ML
models has not been found to give rise to such dis-
sociations (Plaut et al., 1996), and careful analytic
studies of this architecture have led Bullinaria and
Chater (1995) to argue that this is a limitation in
principle (for models that map between only two
types of information).
The debate between proponents of
connectionist and more traditional symbolic (rule-
based) models has tended to become focused on the
properties of SR-ML networks (Pinker, 1997;
Seidenberg & McClelland, 1989). However, we do
not believe that any substantive principle of
connectionist theory is at stake, should such models
be rejected for multiroute alternatives. As noted
above, the SR-ML architecture is a generalisation
of the basic two-layer perceptron, but it is not the
only possible one. To generate the SR-ML net-
work from a two-layer network requires not only
2
The dual-route architecture actually appears in Figures 2 and 3 of the seminal article by Rumelhart et al. (1986), which
introduced the backpropagation algorithm. In particular the first learned solution to the XOR (exclusive-OR) problem (in their Figure
3, p. 331), is shown for a network with two pathways, direct and mediated, between input and output. From there on, however, the
architecture is dropped in favour of the single-route one.
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
(a)
(b) (c)
Figure 2. (a) Simplest feedforward network, consisting of two
layers of units and connections from input to output units. (b & c)
Two generalisations of the two-layer net. (b) single-route
multilayer (SR-ML): Hidden units are added between the input
and output units and the direct connections from input to output
are removed; (c) Dual-route multilayer (DR-ML): As for the SR-
ML net but the direct input-output connections are not removed.
the addition of hidden units and the necessary con-
nections, but also the removal of the existing direct
connections between the input and output layers.
Suppose instead that the hidden units are added but
the direct connections are not removed. This pro-
duces a network with the same number of layers of
units as the SR-ML, but with two pathways from
input to output, one direct and the other mediated
by hidden units (Figure 2)2.
This architecture (call it the dual-route
multilayer network, DR-ML) has a number of
interesting properties that distinguish it from the
SR-ML (Zorzi et al., 1998a, 1998b). For instance,
learning can take place in both routes at the same
time, but the network tends to partition the learn-
ing such that the direct route will learn simple (lin-
ear) regularities, while the mediated route “mops
up” idiosyncratic (exception) input–output pairs by
recognising the exceptional inputs and correcting
the regular response produced by the direct route.
In this case, the network’s ability to generalise to
novel stimuli tends to be concentrated in the direct
route. This means that the mediated route (via hid-
121
 HOUGHTON AND ZORZI
den units) is largely freed from the need to general-
ise and can, in principle, simply learn the training
set on a rote basis. Damage to the two pathways has
different effects and double dissociations of the
type discussed above can be produced. As noted,
the production of such dissociations has proved
extremely challenging for the SR-ML paradigm
(Bullinaria & Chater, 1995; Plaut et al., 1996).
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
A connectionist dual-route model of spelling
The aim of this paper is to implement a detailed
model of the spelling system within the
connectionist DR-ML framework. The model
embodies a number of central theoretical claims.
While some of these are common to dual-route
models, in particular the separation of the whole-
word (lexical) and subword (sound–spelling)
mappings, others are more specific to our model. As
an introduction to the model we describe the main
specific features here:
1. Both routes are activated in parallel with com-
petitive–cooperative interactions at the grapheme (out-
put) level. We propose that a phonological input to
the spelling system will, at least for known words,
activate both a lexical pathway and a phoneme-
based sound–spelling conversion process in parallel.
Both processes produce activation of the same set of
grapheme nodes, and the activations are pooled.
Cooperative interactions take place where both
routes agree on a spelling. Where they do not, posi-
tion-based competition takes place between candi-
date graphemes. In the absence of any random
effects (noise), the grapheme receiving the highest
activation in any position (see below) will be the one
produced.
2. Lexical representation is “localist” (orthogonal
representation of distinct orthographic word forms),
and the rate of activation of lexical nodes depends on
their frequency of use. Although it is not a strictly
required feature of the architecture we use (Zorzi et
al., 1998a), we model the lexical route as a “localist”
network, where each orthographic word form is
represented by a single node (connectionist unit, in
the implementation). We represent the effect of
word frequency at the level of these units them-
122 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
selves (rather than, say, in the connections between
layers), in terms of the strength of positive feedback
a unit provides to itself. The result of this is that
more frequently used lexical items become acti-
vated more rapidly for a given level of input.
3. The sound–spelling mapping is parallel, associa-
tive, and based on experience of whole (real) words.
Where the sound–spelling relationships in a language
are inconsistent, then the representation of phoneme-
grapheme relationships may be one-to-many and
many-to-one. The direct (sound–spelling) route of
our model produces direct associations between
input phonemes and output graphemes. These
associations are generated by training on English
words, whereby the model attempts to predict the
spelling of a word given its pronunciation. The fact
that the associations are direct (implemented as a
two-layer network), means that, when required to
deal with a large corpus of words (over 3000), the
model encodes only the more common spelling–
sound relationships, though these may be inconsis-
tent (that is, after training, multiple candidate
graphemes may be activated by certain phonemes
or phoneme combinations). The scarcity of repre-
sentation resources in direct mapping means that
information about individual lexical items experi-
enced in training is lost when the set is large.
4. Input (phonemic) and output (graphemic) rep-
resentations are syllabically structured, and complex
graphemes (e.g., SH, TH, EE) are locally represented.
We propose that syllabic chunking, at both the
phonological and orthographic levels, is important
to the spelling process (a position shared with other
authors; Caramazza & Miceli, 1990; Treiman &
Chafetz, 1987). The output representation is struc-
tured to use “graphosyllables,” with onset, vowel,
and coda constituents (cf. Plaut et al., 1996), and
also complex graphemes.
The simulations in the paper represent a detailed
empirical test of these proposals against a range of
data from both normal and impaired subjects (sur-
face dysgraphics in particular). In the first section,
we deal with the sound–spelling mapping in isola-
tion, and investigate the model’s nonword spelling
ability following training on a corpus of monosyl-
labic real words. The model’s output is compared to

data regarding normal performance, and the
nonword spelling of a severe surface dysgraphic.
Following this we describe the full dual-route
model, in which the lexical pathway is imple-
mented, and the mode of interaction of the two-
routes is specified. After examining the model’s
“normal” behaviour, we attempt to simulate in
some detail the word spelling of acquired surface
dysgraphics. We model the impairment to the lexi-
cal route as a lowering of the maximum activation
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
level achievable by lexical nodes. Otherwise the
model is unchanged.
A TWO-LAYER NETWORK MODEL
OF SOUND-TO-SPELLING
CONVERSION
Method: Description of the model
3 these proposals.
We begin the discussion of the model by looking at
phonological spelling (sound-to-spelling conver-
sion) in isolation from lexical spelling. We first
develop what we believe is the simplest trainable
mechanism capable of good performance on this
task. Our criteria for “good performance” include
the following: production of phonologically plausi-
ble spellings of nonwords; correct spelling of regu-
lar consistent words; regularisation of exception
words; and ambiguity over the spelling of regular-
inconsistent words (and analogous nonwords).
Representation. The basis of the sound–spelling
model is a standard two-layer feed forward net-
work, with complete connectivity between input
and output units (all output units receive connec-
tions from all input units). The model uses a syllabic
representation for both phonological input and
orthographic output, with a distinction between
onset, vowel, and coda positions. Along with a sin-
gle vowel position, three onset (pre-vocalic) conso-
nant positions and three coda (post-vocalic)
consonant positions are defined, giving a total of
3
The model was implemented in Visual Basic V.5, and runs under MS Windows on an IBM-compatible PC. Enquiries regarding
the implementation should be directed by email to the first author at his current address.
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
seven positions. Within the onset and coda, both
phonemes and graphemes are “left-justified”; that
is, the first phoneme or grapheme occupies position
1, the second position 2, and so on (Figure 3). This
“position-specific” code means that individual
graphemes (especially single consonants) are dupli-
cated, e.g., the onset Rs in RATE and GRATE are
separate nodes, as are the coda Ts in WET and
WEST. That phonological representations are
syllabically organised, with a division between
onset and rime, is widely accepted (Hartley &
Houghton, 1996; Treiman, 1986). The case is per-
haps not quite so clear for orthographic representa-
tions, but a number of authors have argued for the
existence of “graphosyllables” (Caramazza &
Miceli, 1990), and for a division between ortho-
graphic onset and rime (Treiman & Chafetz, 1987;
Treiman, Mullenix, Bijeljac-Babic, & Richmond-
Welty, 1995). The current model is consistent with
In some pilot work, we used an orthographic
representation that lacked complex graphemes
(e.g., SH, DD, OU, etc.), only a single letter being
activated in any position. This produced a rather
low level of plausible spelling (around 70%).
Analysis of the errors showed that the model was
often producing blends of alternative graphemes
for a given sound. For instance, an initial /f/ is usu-
ally spelled F, but can also be spelled PH (phrase,
phase, etc). Without grapheme nodes the model
could sometimes produce the illegal blend “FH.”
Figure 3. Structure of the orthographic output representation.
Seven positions are distinguished, classified as either consonant or
vowel positions. All vowel graphemes (apart from final-e) occur in
the same position. There are three onset consonant positions and
three coda consonant positions. Both onset and coda consonants are
“left justified,” i. e., the first consonant goes in the first slot, the
second, in the second, and so on. (On = Onset, Co = Coda).
123
 HOUGHTON AND ZORZI

Table 1. Complex graphemes used in the model’s output

representation

Onset CH GH GN KN PH QU SH TH WH WR
Vowel AI AIR AR AU AW AY
EA EAR EAU EE EER EI EIR ER EU EW EY
IE IER IEU IEW IR
OA OAR OE OI OO OOR OU OUR OR OW OY
UA UAR UE UI UR UY
YE YR
Coda CH CK DD DG FF GH GHT GN GUE LL
MB NG PH QUE SH SS TCH TH TT ZZ
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

This kind of error indicates that complex

graphemes should compete for output as single
response options. In the above example, the H
should only be produced as long as it is part of a
sequence “PH,” which competes as a unit with the
alternative “F.” Figure 4. Structure of the direct (sound-to-spelling) route of the
The model was thus revised to incorporate com- model. Both phonology and orthography use a syllabic (Onset/
plex graphemes in the orthographic representation Vowel/Coda) representation, and the orthographic representation
(Plaut et al., 1996; Shallice et al., 1995). includes complex graphemes. The figure shows how the input and
output are represented for the mapping from /Tr@Ut/ to
Graphemes are divided into consonant and vowel
THROAT.
graphemes, and all vowel graphemes (with the
exception of final-e) occur in the same position.
The orthographic representation is shown in The phonological input representation is iden-
Figure 3. tical to that used for the output of the reading
The graphemes used (in addition to the standard model by Zorzi et al. (1998a, 1998b). Thus, there
single letters) are shown in Table 1. Table 2 shows are seven phoneme positions defined, one for the
some examples of the way in which individual vowel and three each for pre-vocalic consonants
words are represented in this scheme. With respect (the onset) and post-vocalic consonants (the
to the vowel graphemes, the letters W and R imme- coda). The overall structure of the two-layer
diately following the orthographic vowels are con- model is shown in Figure 4.
sidered to form part of the vowel grapheme (the
model was trained using British English pronunci- Training. The network was trained on a set of
ation, in which post-vocalic /r/ does not occur). 3165 uninflected monosyllabic sound–spelling
pairs. The majority of the words were extracted
Table 2. Examples of the orthographic representation used in the from the Oxford Psycholinguistic Database, and
model these were supplemented by a number of addi-
Onset Coda tional words (in particular, we made sure that all
——————— ——————— the real words required for simulations of experi-
Word 1 2 3 Vowel 1 2 3 mental data occurred in the training set). Regular
three TH R # EE # # # inflected words, e.g., tells, called, cats, were not
itch # # # I TCH # # included, though analogous irregular words were,
hearth H # # EAR TH # # e.g., has, bought, men. Note that the training set
twelfth T W # E L F TH does not define a one-to-one mapping, as it con-
grown G R # OW N # #
tains a number of homophones (e.g., /blu/ →
A # represents an empty position. BLEW, BLUE) and homographs (/lid/, /led/ →

124 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)


LEAD). Prior to learning, the simple ortho-
graphic form of each word is parsed (by a computer
program) into the syllabically-aligned graphemic
representation described above, and associated
with the representation of its pronunciation. Pre-
sentation of the training set occurred in “epochs,”
in which each sound–spelling pair was presented
once, and the order of presentation was random-
ised at the beginning of each epoch (by “shuffling”
the training set used in the previous epoch). Fre-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
quency of presentation was not manipulated; the
extremely short training required by the two-layer
network (learning asymptotes after just a few
epochs, as in Zorzi et al., 1998a, 1998b) would
make a frequency manipulation unrealistic. For
each pair, the required phonological input is set up
by activating the appropriate phoneme nodes to a
value of 1. Activation propagates to units in the
grapheme layer in the usual manner: the
feedforward input to each grapheme unit is the dot
product of the vector of phoneme activations and
the vector of input weights to each grapheme unit
(since the phoneme units have activations of 1 or 0,
this reduces to summing the weights on activated
pathways). Grapheme node activations are then
calculated as a sigmoidal function f of the this input,
bounded in the range [0, 1] and with f(0) = 0 (i.e.,
no input, no output).
The grapheme activation pattern produced in
this way is then compared with the target. Nodes
that should be on have a target activation of 1, while
those that should be off have a target activation of 0.
The error for each output node is the difference
between its target and its actual activation value.
This error is then used to change the weights using
the Delta rule error correcting algorithm (Widrow
& Hoff, 1960), i.e.,
∆w ij = λai (t i − a j ) (1)
where wij is the weight from phoneme unit ui to
grapheme unit uj; ai and aj are the activations of the
phoneme and grapheme units respectively; tj is the
target activation for grapheme unit uj; and λ is a
scaling parameter affecting the size of the weight
changes (conventionally known as the “learning
rate”). The learning rate was typically set to values
between 0.03–0.05 and training stopped when the
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
mean error reached asymptote. This was normally
after around 15–20 epochs of training.
Finally, weight pruning was used to remove
small weights. The delta rule distributes the
“blame” for errors over all activated input pathways,
and early in training, when errors are the norm, this
produces many non-zero weights, which later have
little effect on performance. Pruning “cleans up”
these weights and simplifies analysis of the net-
work. Pruning takes place at the end of each epoch,
and weights whose magnitude falls below some
percentage of the magnitude of the largest weight
(3%–5% in the present simulations) are set to 0.
Note that as training progresses the magnitude of
the largest weights tends to increase, and hence the
absolute value of the pruning threshold also
increases.
Testing. Following training, the model’s perfor-
mance was investigated in a number of studies
reported below. Testing consists of presenting the
phonemic representation at the input, propagating
the activation to the output units (as described
under Training) and then selecting the most active
grapheme at each syllabic position. The selection
mechanism is postulated to be a form of response
competition, and is implemented by lateral inhibi-
tion between the nodes within a given position.
Each node has inhibitory connections to all other
competing nodes, and has a positive feedback con-
nection to itself. After achieving their initial activa-
tion levels due to the phonological input (computed
as during training), the activation level of ortho-
graphic nodes evolves according to the following
rule:
a j (t + 1) = a j (t ) + F j+ (t ) − F j− (t ) (2)
where aj is the activation level of a unit j, t is the time
(in discrete time slices, t = 1, 2, …), F +j is the self-
feedback from a unit j to itself, and F –j is the lateral
inhibitory input from competing units (those rep-
resenting graphemes in the same position). The
two inputs F +, F –j are given by:
w + a j (t ), if a j (t ) > 0
F j+ (t )  (3)
 − ∂a j (t ), otherwise
125
 HOUGHTON AND ZORZI

F j− (t ) = w − ∑ a (t ) i
(4) Results: Performance of the two-layer model
i ∈P ,i ≠ j
Training and basic spelling
where w+ (equation 3) is the positive feedback
Two-layer networks trained with the delta rule are
weight of a node onto itself, and w- (equation 4) is
bound to reach a global minimum with respect to
the lateral inhibitory weight. Both weights were
the total error over the training set. This does not,
set at 0.2. The positive self-feedback only operates
of course, mean that the total error will be 0 (even
if the activation of the node gets above a threshold
less that performance will thereby be optimal with
θ. Below this threshold, the activation decays at a
respect to fitting psychological data). However, in
rate proportional to ∂. The threshold for the
terms of simply minimising the global error, the
orthographic vowel (first rime) position was set
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

weights produced during learning will be about “as

somewhat lower than for the other positions,
reflecting the fact that all words need a vowel
grapheme. The threshold is fixed at 0.1 for the
vowel position, and at 0.2 for all other positions.
Equation 4 expresses the lateral inhibitory input to
a node j; the summation is over all other nodes i ≠ j
in an orthographic position P. Finally, the activa-
tions of output nodes are bounded (clipped) in the
range [0,1].
To summarise: During testing, the two-layer
model’s response to a phonological input is gener-
ated as follows:
1. The phoneme nodes comprising the input
word or nonword are set to an activation value of 1.
2. This activation propagates through the
learned weights producing an initial orthographic
activation (sigmoidal function of net input).
3. Phonological input then ceases (the word
does not stay in the environment) and the activated
orthographic nodes obey equations 2–4 until all
output nodes reach a steady-state activation of
either 1 (fully on) or 0 (full off).
4. The model’s spelling of the input word is that
specified by the set of grapheme nodes with an acti-
vation of 1 eventually produced by step 3. A mea-
sure of relative “reaction time” to generate a
coherent spelling plan is provided by the number of
cycles needed for the orthographic nodes to settle
down.
good as it gets” for the given architecture. Unlike
the case of models containing hidden units, there is
no scope for optimising the model with respect to
additional performance criteria; the network can-
not be hand-crafted (by varying the number of hid-
den units) to produce a desired a level of
generalisation, or otherwise fit known data4.
Nevertheless, there is some room for variation in
the model in terms of, for instance, the learning rate
used, the pruning threshold, and the length of
training. To examine this, we have trained the
model many times using different combinations of
these learning parameters, within limited ranges.
For none of the model behaviours reported below
have we observed any major effect of this variation,
though the model’s spelling of some stimuli may
alter, and overall response times vary slightly.
Therefore, in all the simulations reported in the
paper, we use the same set of weights in the phono-
logical route. These were produced by training the
model for 20 epochs, with a learning rate of 0.03,
and a pruning threshold of 5%. At 20 epochs the
model’s error curve has been more or less flat for a
number of epochs, i.e., further improvement on the
training set is negligible. The learning rate is low
enough to minimise any potential “recency” effects
arising from the last words the network happened
to be trained on, but not so low as to prolong train-
ing times unnecessarily.

4
For instance, Plunkett and Marchman (1991) used 20 hidden units in a three-layer model for learning the past tense of English.
They explain this number as a result of experimentation in which they varied the number of hidden units between 10 and 120. The final
choice of 20 is said to represent a “compromise” between optimal performance and the desire to maximise the network’s ability to
generalise. Similarly, Hahn and Nakisa (2000) describe results for a multilayer perceptron model for producing the plural of German
nouns. They report having experimented with networks with hidden units ranging from 2–1000. However, they only report the actual
results from a single network (the best performing one, with 40 hidden units).

126 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)

 CONNECTIONIST DUAL-ROUTE SPELLING

Table 3. Model's spelling of some of the exception words in its such subjects are the least likely to be able to use lex-
training set ical information. We would repeat the point made
Correct Model Correct Model above, that since this route has no hidden units,
there is no possibility of varying the number of
SOAP SOPE NIGHT NITE
SCHOOL SCOOL SWORD SORD them until a desired degree of generalisation is
AISLE ILE CHIC SHEEK obtained.
COUGH COF LIMB LIM Table 4 provides a comparison of the model’s
spelling of a set of 58 nonwords with those pro-
duced by MP, a surface dyslexic and dysgraphic
After training, the model’s spelling was first subject who showed near-perfect performance on
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

tested on all of the words in the training set, using
the procedure set out above. The model spelled
about 68% of the words correctly. This compares
with a figure of 81% correct in reading a similar
(though slightly smaller) training set by the related
reading model of Zorzi et al. (1998a). This differ-
ence reflects the greater inconsistency of the Eng-
lish sound-to-spelling mapping compared to the
reverse. For regular consistent words (plank, match,
bill, etc.) the model’s spelling was correct, while
exception and inconsistent words were regularised
(see Table 3 for examples). This aspect of the
model’s performance is investigated more system-
atically below. Overall, 96% of the words’ spellings
were judged to be phonologically plausible, in the
sense that the pronunciation of the word could be
reliably retrieved from the spelling. Many of the
errors involved adding an unlikely final “e” on some
words, so for instance, /swIs/ was spelt “swisse”.
Although these were scored as errors, we suspect
many would be pronounced correctly.
Simulation 1: Nonword spelling in surface
dysgraphia
Of particular interest is the model’s nonword spell-
ing (generalisation). Since the isolated two-layer
model has no lexicon, it cannot spell nonwords by
lexical analogy and must use any sound–spelling
regularities it has extracted. An interesting test of
the model is therefore to compare its nonword
spelling with that of a surface dysgraphic subject, as
nonword reading and spelling (Behrmann & Bub,
1992).
The spellings produced by the model are all
plausible, and in many cases identical to those pro-
duced by MP. One possible criticism is that the
model sometimes produces a redundant final-e,
e.g., in moope. This is due to the fact that “silent e”
spellings, as in late, pile, mole, etc., do not compete
with alternative spellings as a single response
option. Hence the final-e can be selected irrespec-
tive of the chosen vowel (Houghton & Zorzi,
1998). There is evidence that literate adults also
produce such spellings. For instance, most subjects
in the Barry and Seymour (1988) study spelled the
pseudoword /teIn/ either tane or tain. However a
few spelled it taine or tayne (redundant final-e).
Similarly, the stimulus /pim/, generally spelled
peem or peam, was sometimes spelled peeme or
peame.
It is noticeable that the model produces spellings
identical to those of MP where the sound-to-spell-
ing mapping is highly consistent, e.g., dimp, wush,
fent, cham, etc. In the case of inconsistent mappings,
the model may generate an alternative acceptable
spelling, e.g., dreece vs. dreace, kail vs. caile, caum vs.
corm5. In most of these cases, we find that the spell-
ings produced by MP have in fact been activated in
the model but have not won the output competi-
tion. For instance, Figure 5 shows the initial
grapheme activations (prior to response resolution
by the lateral inhibition mechanism) generated by

5
The difference between caum (MP) and corm (model) (similarly for bauce vs. borse) is probably due to dialect. MP is American and
we assume would not spell a pseudoword with a post-vocalic R unless there was an /r/ (or roticised vowel) in its pronunciation (hence
caum, bauce; cf. Treiman & Barry, 2000). The model, however, is trained on (Southern) British English pronunciation, in which the
dominant spelling of the long vowel /O/ is OR. The model’s spelling is supported by the Barry and Seymour (1988) study, in which
about 80% of the (British) participants produced OR when spelling the nonwords /vOk/ and /nOn/ (see also Treiman & Barry, 2000).

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 127

 HOUGHTON AND ZORZI

Table 4. Nonword spelling: comparison of model with surface dyslexic MP (Behrmann &
Bub, 1992, Appendix 2)

Word MP Model Word MP Model

/bil/ beel beel /bl@Um/ blome blome

/sek/ sek seck /kOm/ caum corm
/mel/ mell mel /tS&m/ cham cham
/sp&l/ spal spal /hin/ heen heen
/ked/ ked ced /ditS/ deetch deech
/fleIt/ flate flate /dreIt/ drate drate
/neld/ neld neld /d0ld/ dold dold
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

/k@Ub/ cobe cobe /fl&m/ flam flam

/bOs/ bauce borse /l&ts/ lats lats
/dImp/ dimp dimp /sVf/ suff suff
/w@Ul/ wole wole /wVS/ wush wush
/lim/ leem leam /dris/ dreece dreace
/seIt/ sate sate /sk3l/ skirl scurl
a
/k0dZ/ cudge codge /nitS/ neech neech
/frIm/ frim frim /swim/ sweam sweam
/nIlt/ nilt nilt /mup/ moop moope
/trIst/ trist trist /teIz/ taze tase
/flip/ fleep fleep /hoIl/ hoil hoil
/blim/ bleam bleam /bris/ brease breace
/ged/ ged ged /witS/ weech weech
/rIlt/ rilt rilt /plik/ pleak pleak
/ris/ reese rease /hoIs/ hois hoise
/maIz/ mise mise /keIl/ kail cale
/fent/ fent fent /degz/ degs degs
/rel/ rell rel /st0ld/ stold stold
a
/pl0k/ pluck plock /skIm/ skim skim
/r0g/ rog rog /petS/ petch petch
/blIk/ blick blick /l0t/ lot lot
/fritS/ freech freech /nud/ nood noode
a
Denotes response marked as an error by Behrmann and Bub.

the input /dris/. The grapheme EE (produced by
MP) has an initial activation of just over 0.3, which
is later suppressed in the model by the more active
EA grapheme.
However, this does not mean that the model will
always spell /i/ as EA, even in the absence of any
random factors (noise). As can be seen in Table 4,
the model’s spelling of /i/ actually varies. In the
pseudowords /bil/, /ditS/, /flip/ it is spelled EE
(beel, deech, fleep), but is spelled EA in /swim/, /bris/,
and /plik/ (sweam, breace, pleak). Interestingly, MP
shows very similar variation (compare the model’s
and MP’s spelling of these words in Table 4). This
shows the influence in the model (and presumably
in MP) of surrounding phonological context on the
selection of inconsistent phoneme–grapheme
mappings. We return to the issue of the
representation of inconsistent sound-to-spelling
relationships, and the effect of phonological con-
text, below.
Simulation 2: Consistency effects in word spelling
We noted earlier that the model tends to spell regu-
lar words correctly and regularises inconsistent and
exception words. To make a more systematic com-
parison between the model’s spelling of consistent
and inconsistent words and nonwords, we used
word lists produced by Stone, Vanhoy, and Van
Orden (1997) based specifically on their sound-to-
spelling consistency. Some words that have a con-
sistent pronunciation with respect to their spelling,
e.g., “bean”, may be inconsistently spelled, given

128 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)


Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Figure 5. Initial orthographic response to nonword input /dris/,
showing competition between two possible renderings of the vowel
/i/. Y-axis shows activation level, x-axis shows grapheme output
position. Dark bars represent the most active response in each
position; lighter bars competing responses. Competitive interactions
within the vowel position lead to the less activated option EE
being suppressed. Note also the (subthreshold) activation of S in the
first coda position. (On = Onset, Co = Coda).
Figure 6. Comparison of the two-layer model’s spelling of words
showing consistent versus inconsistent sound-to-spelling patterns.
Graph shows per cent correct vs. incorrect (but plausible) spellings
of words of each type. All tested words were in the model’s training
set.
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
the pronunciation, i.e., the syllable /bin/ could be
spelt bean, been, or even bene. Words such as plank,
march, loft, however, are considered sound-to-
spelling consistent, as there is (putatively) no other
plausible way to spell them (see Ziegler et al., 1997).
If the model has indeed extracted the basic regulari-
ties of the sound–spelling mapping for English,
without remembering the specific words it was
trained on, it should show a clear difference
between these sets of words, spelling the consistent
words the way they are actually spelt, but producing
alternative spellings for more of the inconsistent
words.
The model was presented with the phonological
forms of two lists of words, consistent and inconsis-
tent, from the Stone et al. (1997) Experiment 1. All
the words in both lists occurred in the model’s
training set. The model’s spellings of the words
were recorded and compared to the actual spelling.
The results are shown in Figure 6.
Of the 25 Stone et al. (1997) consistent words,
the model spelled 23 of them correctly (92%). The
two “errors” were fluke spelled flook, and loaf spelled
loafe, both of which we consider phonologically
plausible alternatives. Of the 25 inconsistent words,
the model spelled only 6 correctly (24%), while all
25 spellings were phonologically plausible. The
model thus shows clear sensitivity to the dominant
sound–spelling regularities of English when spell-
ing words on which it has been trained. If the word
is an inconsistent or an exception word this could in
principle act as a source of interference with respect
to lexical (case-specific) spelling. This issue is
investigated in detail in later simulations.
Simulations 3 and 4: Reaction times in the assembly
route
As shown above, the model can generate alternative
spellings where there is phoneme–grapheme
inconsistency in the training set. As well as predict-
ing variability in spelling, the time taken by the out-
put mechanism to resolve this response
competition can be used to predict reaction time
differences as a function of sound–spelling consis-
tency. We can show the effect of consistency on
reaction time by looking at the time the model takes
to produce a stable output representation in
129
 HOUGHTON AND ZORZI
response to the different types of Stone et al. stim-
uli. These stimuli also allow us to make a word–
nonword comparison orthogonal to consistency.
Of course, with respect to spelling real words, we
cannot directly compare the model’s RT to human
data, as normal participants may not spell words
(especially inconsistent and exception words) using
an assembly procedure. It is nevertheless of interest
to compare the model’s word and nonword spelling
to see whether the assembly route shows any kind of
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
lexicality effect—that is, does having been trained
on a particular word speed up processing of that
word compared to nonwords controlled for
consistency?
Simulation 3. We examined the latency times for
the four word lists from Stone et al. (1997), Experi-
ment 1. These lists comprise consistent and incon-
sistent words and nonwords. There are 25 words in
each list. The four lists were presented to the model
and the time taken (in simulated processing cycles)
to produce a stable response to each word was
recorded. The means for each condition are shown
in Figure 7. The results were analysed with a two-
Figure 7. Mean time taken by the model (in processing cycles) to
produce a stable and unambiguous orthographic response as a
function of stimulus type: Factors are lexicality (word vs.
nonword) and consistency (consistent vs. inconsistent). Only the
effect of consistency is significant.
130 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
way analysis of variance (ANOVA) in which the
factors were lexicality (word vs. nonword), and con-
sistency (consistent vs. inconsistent). A significant
main effect of consistency was found—the model
takes longer to settle on a spelling for inconsistent
stimuli than for consistent ones, F(1, 96) = 18.04, p
< .001. However, neither the lexicality factor nor
the interaction between the two factors was signifi-
cant. The absence of a lexicality effect means that,
at least in terms of response times, the phonological
route does not distinguish between words it has
been trained on and novel stimuli. This is an impor-
tant property of the model which we will return to
later in the context of a simulation of surface
dysgraphic spelling errors.
Simulation 4. We wished to confirm the consistency
effect found for words on which the model has been
trained, as this is important for understanding the
interaction between the lexical and phonological
routes studied later in the paper. For this purpose,
we presented the model with sets of regular consis-
tent and exception words taken from the Taraban
and McClelland (1987) study of consistency effects
in reading. The regular consistent words had to be
altered slightly, as they contained words that were
somewhat inconsistent from a spelling point of
view, e.g., write, which. Some other words were
changed as they were inflected, and hence were not
in the model’s training set. The resulting stimulus
lists used are given in Appendix A. In all there were
48 regular words and 48 exception words (all in the
model’s training set). The model’s spelling of each
word was recorded, along with the time taken (in
number of cycles). Only three regular words were
incorrectly spelled, whereas most of the exception
words were regularised (37/48, 77%). The signifi-
cant consistency effect was also confirmed in the
response latencies. Mean response time for the
regular words was 4.33 cycles, while the exception
words (even though regularised) needed 6.9 cycles,
t(96) = –4.11.
Simulations 5 and 6: Phoneme–grapheme
consistency and the effects of context
In Simulation 1, it was noted that the model can
spell a given vowel sound (e.g., /i/) in more than one
 CONNECTIONIST DUAL-ROUTE SPELLING

way, apparently dependent on phonological con-
text. On the other hand, the model misspells excep-
tion words it has been repeatedly trained on
(Simulations 2–4), indicating that it does not use
the whole-word context to generate a correct spell-
ing. In such cases it appears to be largely driven by
regular phoneme–grapheme correspondences. In
this section we investigate these properties in more
detail, and simulate nonword spelling data from
normal subjects.
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
ing. Here we confine ourselves to examples that
illustrate the way in which the model responds to
differences in the degree of consistency of a
phoneme–grapheme relationship, and its use of
context.
Figures 8 and 9 each show the excitatory con-
nections from one group of phoneme nodes to one
group of grapheme nodes. Figure 8 shows the con-
nections from the syllable-initial consonants to the
syllable-initial graphemes, and Figure 9 those from

the phonological to the orthographic vowels. The

Connection weights following training. Before relative strength of the connection from a given
describing the results of the simulations, it is phoneme to a given grapheme is shown by the size
instructive to examine some aspects of the pattern of the filled rectangle in the relevant cell of the
of weights formed by the model as a result of train- matrix.

Figure 8. A subset of the weights developed by the model during learning. The matrix shows the excitatory weights formed between phoneme
nodes in the first onset position (rows) and grapheme nodes in the equivalent orthographic position (columns). Each row and column is
labelled with the associated phoneme and grapheme respectively. The size of a filled rectangle indicates the (relative) strength of the
connection between the nodes in the corresponding row and column. Empty cells have zero weights.

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 131

 HOUGHTON AND ZORZI
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Figure 9. Excitatory weights formed between phonological and orthographic vowel nodes during learning. See Figure 8 for explanation and
comparison.
The initial consonants (Figure 8) show a clear
pattern. Most phonemes have a strong excitatory
connection to just one grapheme. In a few cases, a
phoneme may show a second connection, e.g., ini-
tial /k/ has connections to both C and K. It is evi-
dent that where a particular phoneme–grapheme
mapping is highly regular, and independent of con-
text (e.g., words with an initial /b/ are always spelled
with a B), the model simply learns a strong excit-
atory connection between the relevant (position
specific) phoneme and grapheme. Otherwise, few
connections develop to such graphemes. Exam-
ining the weight matrix as a whole we find that, in
the case of initial consonants such as B, P, D, M,
etc., the large excitatory weight shown in Figure 8 is
the only such connection these graphemes receive.
Hence, in these cases the model acts as a simple
phoneme–grapheme conversion mechanism.
When phoneme–grapheme relationships are
not so straightforward, the model will attempt to
use any information from the phonological context
132 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
which can help it make a correct choice. For
instance, after short vowels (/I/, /&/, etc.) a final /k/
is usually spelled CK (SICK, BACK, LOCK, etc.),
whereas after long vowels and diphthongs, it is
spelled K (WEAK, FORK, CROAK, etc.). The
model correctly predicts the spelling of the final /k/
in these words (and also for nonwords derived by
changing the initial consonants of the words). This
ability must depend on connections from the pho-
nological vowel to the relevant coda consonants.
Inspecting the relevant connections (not illustrated
here), we find first that the coda /k/ has excitatory
weights to both K and CK. Looking at the weights
from the vowel phonemes, we find that the set of
short vowels have excitatory connections to CK,
and strongly inhibit K. The other vowels simply
inhibit CK. Hence the /k/ activates both candidates
and the vowels select between them.
Turning to the vowel-to-vowel connections
(Figure 9), it is immediately apparent that they
show much more ambiguity than the consonants.

Most vowels have connections to more than one
graphemic vowel and many of the connections are
relatively weak (the scaling of the rectangles repre-
senting connections in Figures 8 and 9 is the same).
It is noticeable, however, that the short vowels (/I/,
/e/, /&/, /0/, /V/, /U/) have relatively unambiguous
connections, each forming a dominant link to a
single grapheme (I, E, A, O, U, OO, respectively).
The spelling of these vowels is generally consistent.
The long vowels and diphthongs show more ambi-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
guity. For instance, earlier we noted the model’s
variability in spelling the long vowel /i/ (EE or EA).
In Figure 9 we can see that this vowel has similarly
sised connections to EE and EA, with a slight
advantage for EE. In such cases, grapheme selec-
tion can easily be influenced by the balance of excit-
atory and inhibitory inputs from the phonological
context.
We now describe two simulations aimed at
assessing the model’s weighting of vowel spellings,
and its use of phonological context to determine
which vowel is the most appropriate.
Simulation 5. We can get some measure of the
appropriateness of the ambiguous connection pat-
terns formed by the vowels by comparing them with
data on variability in vowel spelling in nonwords
provided by Barry and Seymour (1988). This study
was mainly aimed at investigating lexical priming of
vowels in nonword spelling (the spelling of incon-
sistent nonwords can be affected by lexical primes),
but unprimed spellings were collected to provide a
baseline. Each vowel was tested using two
nonwords (e.g., for the vowel /eI/, the stimuli were
Table 5. Ambiguity in vowel spelling by participants and the model
Long vowels
—————————————————————
Vowel Participants Model
/i/ EE > EA EE > EA
/u/ OO > U-E OO > EW > U-E
/O/ OR > AW > AU OR > AW >AU
/3/ UR > ER UR > IR > ER
a
Participant spellings are ranked according to frequency of use (data from Barry & Seymour, 1988, Appendices 1 and
2), model spellings by strength of connection from vowel phoneme to graphemes. Spelling with a dash (e.g., O-E)
denotes final-E spelling (e.g., POVE).
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
/teIn/ and /pleIl/). Appendices 1 and 2 of Barry and
Seymour (1988) list all the unprimed spellings
produced by their subjects for each nonword, along
with the (absolute) number of subjects producing
each spelling. From these raw scores (combining
the data from the two nonwords for each vowel) we
can produce a rank ordering of the frequency of use
of a particular vowel spelling in this study. This
ordering can be compared with the relative strength
of the connections in the model from the same pho-
nological vowels to vowel graphemes (Figure 9).
Of the 13 long vowels and diphthongs repre-
sented in the model, the Barry and Seymour (1988)
study provides data on the spelling of 9. All 9
showed variability in their spelling. Table 5 shows
the graphemes used to spell these vowels in this
study, ranked (left to right) according to frequency.
(Only the two or three most frequent spellings are
listed. These account for the vast majority of the
data. Individual subjects occasionally produced
spellings that we have not listed.) Alongside the
subject spellings, we list the two or three vowel
graphemes which receive the strongest inputs from
the phonological vowels in the model (Figure 9), in
this case ordered according to strength of connec-
tion. As can be seen, the variability inherent in the
model connectivity closely matches the variability
in the subject data. In most cases, the spellings the
model activates correspond to those used by the
subjects, and the rank orderings are very similar.
Simulation 6. As noted, the major locus of inconsis-
tency in English orthography is in vowel spelling
(see Kessler & Treiman, 2001, for a quantitative
a
Diphthongs
—————————————————————
Vowel Participants Model
/eI/ AI > A-E > AY A-E > AY > AI
/aI/ I-E > Y(-E) I-E > Y(-E)
/oI/ OI > OY OI > OY
/@U/ O-E > OA > OU O-E > OA > OW
/aU/ OW > OU OU > OW
133
 HOUGHTON AND ZORZI

demonstration of this). However, statistical analy-
ses of English sound-to-spelling correspondences
have shown that the spelling of some inconsistently
spelled vowels becomes more predictable when the
preceding (onset) or following (coda) consonants
are taken into account, though the effect of the coda
consonants is considerably larger (Kessler &
Treiman, 2001). While we were in the process of
revising this article, we became aware of the results
of a study by Treiman, Kessler, and Bick (in press,
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
to conclude that adult spellers are influenced by the
surrounding phonological context when spelling
vowels.
To test the model against these results we pre-
sented it with the nonword stimuli used in the
Treiman et al. (in press) Experiment 1. As well as
looking at the actual spellings produced by the
model, we also compared the initial levels of activa-
tion of both the critical and default spellings in the
experimental and control contexts. This is a useful

Experiment 1), who tested whether adult spellers
showed any sensitivity to the phonological context
provided by the coda when spelling monosyllabic
nonwords. In this study, for each vowel that was
investigated, two different phonological contexts
were identified such that a particular spelling,
referred to as the “critical spelling,” is used more in
one context than the other (in the set of English
monomorphemic monosyllables). For instance, for
the vowel /e/ (as in PET), the critical spelling is EA,
which commonly occurs only in the coda context /
_d/ (e.g., HEAD, SPREAD, DREAD, etc). In
other contexts the spelling is consistently E, and
this was the spelling expected to be produced in a
set of control nonwords (we will refer to this as the
“default spelling”, to contrast it with the critical
spelling). For each such case, Treiman et al. con-
structed 10 nonwords with the experimental con-
text (i.e., that associated with the critical spelling)
in the coda. Each experimental stimulus was paired
with a control stimulus, which was identical except
that it lacked the experimental context. Treiman et
al. report that the rate of production of the critical
spellings was significantly greater in the experimen-
tal than in the control contexts, leading the authors
comparison, as it provides a more detailed picture of
the relative availability of a given spelling in a given
context. It also permits us to perform a statistical
analysis by item of the difference between the con-
trol and experimental contexts.
In their Experiment 1, Treiman et al. (in press)
investigated the effect of coda consonants on the
spelling of six vowels /e/, /i/, /U/, /aU/, /@U/, and
/aI/. Of these we were able to test the first four in
the model, as they involve a simple comparison
between different vowel graphemes6. The results of
the simulation and a comparison with the Treiman
et al. data are shown in Table 6. The first two rows
show the proportion of words produced with the
critical spelling in the experimental and control
contexts, by both the human subjects and the model
(the human data are the means across subjects). As
can be seen, for each of the four vowels the model
produced more of the critical spellings in the exper-
imental contexts. Interestingly, for three of the four
vowels, the model’s rate of production of the critical
spelling was quite close to the mean of the subjects.
The exception was the vowel /U/ (critical spelling
U, expected spelling OO). The model was rather
less likely to produce the critical grapheme than the

6
We did not include /@U/ because the comparison is between the critical spelling O and expected spelling O_E. The latter (a
silent-E spelling) is not a grapheme in the current model, and we cannot assess its activation level independently of the activation of O
in isolation. For the case of /aI/ the experimental context was /_t/ and the critical spelling IGH, as in e.g., N-IGH-T. That is, Treiman
et al. (in press) effectively treat IGH as vowel grapheme and contrast it with the expected spelling I_E. Our model does not concur with
this analysis; instead we parse NIGHT as N-I-GHT, with GHT as a coda grapheme. The control nonwords used by Treiman et al. for
this vowel had codas other than /t/, e.g., /draIb/, /gaIf/, and hence the control condition measured the rate at which subjects produced
spelling such as DRIGHB and GIGHF. Our model simply predicts that such spellings will never be produced, as they are
orthographically impossible for it. In agreement with this, Treiman et al. report that such spellings were never produced by any subject.
For the experimental nonwords (with rime /aIt/), the model spelled 1 of the 10 words with IGHT, and activated the GHT grapheme to
some degree in every case (which it did not do for the control nonwords). Treiman et al. report a mean proportion of .27 for the critical
spelling in the /_t/ context. Hence the model behaviour is consistent with the results for this vowel, although the complete explanation
it offers of the difference between the conditions involves the orthographic representation, which is not the issue being addressed here.

134 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)

 CONNECTIONIST DUAL-ROUTE SPELLING

Table 6. Results of simulation 6. Data from Treiman et al. (in press) Experiment 1

/e/ – EA /i/ – EE /U/ – U /aU/ – OW

——————– ——————– ——————– ——————–
Vowel—Critical spelling Data Model Data Model Data Model Data Model

Proportion of critical spellings,

experimental nonwords .11 .10 .72 .90 .54 .20 .50 .60
Proportion of critical spellings,
control nonwords .05 .00 .45 .50 .43 .10 .08 .00
Mean proportion of vowel activation
due to critical spelling, experimental nonwords .19 .63 .42 .50
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

Mean proportion of vowel activation

due to critical spelling, control nonwords .10 .45 .29 .27
Difference in critical vowel activation,
experimental vs. control, t test by items t(9) = 2.49 t(9) = 2.95 t(9) = 4.04 t(9) = 4.36

human subjects, irrespective of the context, prefer-
ring OO in nearly every case.
As a more sensitive test of the effect of context,
we examined the relative availability of the critical
grapheme by comparing its activation to that of the
default grapheme for each stimulus. This was done
by expressing the activation level of the critical
grapheme as a proportion of the summed activation
of both the critical and default graphemes—for-
mally, aprop = acrit/(acrit + adef), which is equivalent to
the Luce choice rule for two responses. The means
of this measure for each vowel for the experimental
and control nonwords are shown in rows 3 and 4 of
Table 6. Row 5 shows the results of t tests by items
comparing the two conditions. For every vowel, the
relative activation of the critical grapheme is signif-
icantly higher in the experimental than in the con-
trol context.
Discussion of the phonological-route model
We have shown above that a two-layer network,
trained on whole-word sound–spelling pairs using a
very general learning rule (Sutton & Barto, 1981),
comes to behave like a sublexical sound-to-spelling
conversion system. That is, in the process of trying
to learn to spell words, it acquires all the functional
characteristics traditionally associated with the
phonological or assembly route of the dual-route
framework. Specifically,
1. Nonwords are spelled plausibly, generally
using the most frequent sound–spelling relation-
ships. However, at the level of phoneme–grapheme
mappings, the model shows some variability.
2. Regular-consistent words are spelled
correctly.
3. Exception words are regularised.
The phonological route shows additional proper-
ties that are not necessarily associated with the
dual-route framework, in particular,
1. Multiple grapheme candidates can be acti-
vated in parallel, resulting in response competition.
2. This causes variation in response latencies,
resulting in a consistency effect—inconsistent
words and nonwords take longer to spell.
3. Despite the model only being trained on
words, there is no effect of lexicality on spelling
latencies (and no interaction of lexicality with
consistency).
4. Individual phonemes are not always spelled
in the same way, but show effects of surrounding
context. The model is not functionally equivalent,
for example, to a set of context-free phoneme–
grapheme mapping rules.
With respect to the three traditional properties of
the phonological route, we emphasise that they are
not due to any nonstandard mechanisms built into
the model. The model is a two-layer feedforward
neural network, trained using a general learning
rule, one which has found quite wide application in

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 135

 HOUGHTON AND ZORZI
human and animal learning (Gluck & Bower,
1988a, 1988b; Miller, Barnet, & Grahame, 1995;
Shanks, 1991). This model minimises its error on
the words it is trained on, but due to limitations
inherent in the architecture, it is incapable of reduc-
ing the error to 0 on this particular problem.
Instead, it can only represent the more statistically
reliable sound–spelling regularities. These regulari-
ties also turn out to produce good nonword spell-
ing. This is an especially significant result in a two-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
layer network, because the model cannot be tuned
post hoc to produce a desired degree of generalisa-
tion, for instance by trial-and-error selection of the
number of hidden units (see Footnote 4).
With respect to the second set of properties
listed above, the central feature of the model is that
it need not generate just a single candidate
grapheme for each phoneme, but can produce mul-
tiple competing candidates. In addition, the relative
strength of these candidates is not fixed, but is
affected by surrounding context. This property is
crucial to explaining the kind of variability found
in subject data on nonword spelling (Barry &
Seymour, 1988; Treiman et al., in press), and
depends on the weights formed during learning.
Having established the viability of one route of
the dual-route multilayer net as a sound-to-spelling
conversion mechanism, we now turn to the other
route, and to the interaction between the routes at
the common output level. This second route
enables interactions between sound and spelling to
be mediated by another layer of units. In the follow-
ing we will assume that these units simply learn the
patterns they are exposed to, i.e., that they function
as an orthographic output lexicon.
A PARALLEL, INTERACTIVE
MODEL OF SPELLING
In the simulation of the isolated assembly route
described above, there is no temporal dynamic in
the way the phonological input initially activates
the graphemic output. Phonological activation
spreads in a single pass to the grapheme nodes,
which then compete to generate an unambiguous
spelling plan. This permits the effects of response
136 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
competition to be studied with a minimum of
complicating factors. However, we now want to
consider the temporal interaction of this route with
output from a lexical procedure. As described in
more detail below, we envisage the build-up of lexi-
cal activation as being a gradual process; in particu-
lar, we will argue that the spelling of high-
frequency words becomes available more rapidly
than that of low-frequency words (see Zorzi et al.,
1998a, for a similar treatment of lexical activation in
reading). To model the interaction of the phono-
logical route with this process, we also require its
output to build up over time, rather than being
instantly available.
Method: Description of the model
We will describe the functioning of the two-route
model in three stages: first, the activation of
graphemes by the lexical route; second, the
activation by the phonological route; and finally,
the integration of the two sources of activation.
Dual-route model: Grapheme activation by the
lexical route
To model lexical contributions to spelling, we have
implemented a simulated orthographic output lexi-
con (OOL). The OOL itself is conceived of as a
competitive, localist network in which each node
represents the spelling of a single orthographic
word form, and competitive interactions allow just
one node to become active when a familiar phono-
logical code is encountered. The implementation of
the lexical route involves a number of architectural
simplifications compared to box-and-arrow models
such as that shown in Figure 1. The main functional
property we are interested in implementing is the
gradual activation of a single orthographic lexical
candidate. In principle this might be due to input
from a phonological lexicon, semantics, or both.
However, we do not attempt to simulate these pro-
cesses. In the implemented model, the vector of
phoneme activations is simply matched against a
stored input weight vector for each orthographic
lexical item, and the word producing the best match
between the two is selected. If the input word is a
nonhomographic homophone (e.g., /bin/, BEEN/
 CONNECTIONIST DUAL-ROUTE SPELLING

BEAN), then the most frequent spelling is selected
(we assume that under normal circumstances
semantic context plays a part in disambiguating the
input).7 Thus the two important features of this lex-
ical process are:
1. Activation of a selected lexical node is grad-
ual, building to an asymptotic level over a number
of time steps.
2. The rate at which a lexical node becomes
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
where w+i is the feedback weight to a lexical unit ui,
Wmax is the maximum possible weight, freqi is the
Kucera and Francis (1967) frequency count of the
word, and f (.) is a sigmoidal function in the range
[0,1], with f (0) = 0. The parameter Wmax is set to .9
in all the following simulations. The way in which
this frequency-dependent parameter affects the
“excitability” of a unit is described next.

activated by its adequate stimulus depends on the

frequency of the word it represents.
We start with the frequency manipulation.
Word frequency as a dynamic effect. The nodes in the
OOL are conceived of as being embedded in a com-
petitive network of the type discussed above for the
grapheme output process. In such a network each
node has an excitatory feedback loop onto itself,
giving it the ability to support its own activation.
Typically, the strength of this feedback depends on
a parameter (the feedback weight) that is the same
for all nodes in the network (see equation 3). In the
implementation of the OOL, we allow this feed-
back weight to vary as a function of word fre-
quency—the more frequent a word is, the stronger
the feedback weight. In principle, we believe that
such a modulation of a unit’s feedback weight could
easily be achieved as part of a competitive learning
algorithm. In the implementation we simply derive
the feedback weight for each unit from its frequency
by a rule, in which a maximum possible value for the
weight is multiplied by a sigmoidal function of
frequency:
w i+ = W max ∗ f ( freq i )
Activation of the lexical nodes. The input to a lexical
node from the input phonemes is assumed to
develop gradually, reaching an asymptotic level in a
given number of time steps, a process we refer to as
“ramping.” The phonological input to a lexical node
ui at time t is given by,
Input i (t ) = Phon max ∗ α [ ramp − t ] (7)
+
where Phonmax is the maximum possible input, α is a
scaling factor, ramp is the time it takes (in discrete
cycles) for the input to reach its maximum value,
and [x]+ = max(0,x). In all simulations Phonmax = 1, α
= .8. The value of the parameter ramp is discussed
below.
The activation level LexAi of a lexical node ui at
time t is then given by the sum of its current input,
as defined by equation 7, and a frequency-depend-
ent proportion of its previous activation,
LexA i (t ) = Input i (t ) + w i+ LexA i (t − 1) (8)
The value of the feedback weight w+ depends on
the word frequency as specified in equation 6.
Finally, lexical node activations are bounded in
[0,1]. The overall result of equations 6–8 is simply
that the selected lexical node becomes activated
gradually, and that the rate of increase of activation
(6) is proportional to its frequency.

7
A further consequence of the simplification of access to the OOL is that partial activation of nodes representing words
phonologically similar to the target does not occur. However, we do not wish to claim that partial activation of OOL nodes is
impossible. As stated, we envisage all lexical nodes as being embedded in a competitive network in that lateral inhibitory interactions
allow the node that best “matches” current inputs to dominate the activation of other nodes, i.e., precisely the kind of dynamics
implemented amongst the grapheme units which allows selection of at most a single response in each syllabic position. While there is
no theoretical difficulty in implementing the competitive selection mechanism amongst the lexical nodes, we felt that the additional
complexity it would add to the implementation, including the need for further free parameters governing the competitive interactions,
would have little benefit with respect to the phenomena we are interested in, as we do not attribute any of them to possible activation
effects of multiple lexical candidates.

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 137

 HOUGHTON AND ZORZI
Lexical activation of grapheme nodes. A lexical node
sends activation to all graphemes that it contains
and inhibition to those it doesn’t. Thus the lexical
input, LexInput, to a grapheme node i given by,
LexInput i (t ) = w lex LexA j (t )
where LexA is the activation of the selected lexical
item (equation 9), and wlex is a weight parameter.
For graphemes that are in the word, wlex = 1.0; for
those which are not wlex = –.5. The positive weight
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
simply transfers the lexical activation directly to the
graphemes contained in the word, while the nega-
tive weight helps to suppress any inconsistent input
from the sound–spelling route. The more rapid
activation of higher-frequency words leads to more
rapid activation of their constituent graphemes. As
noted above, lexical node activation has a maximum
value of 1.0. The setting of the excitatory lexicon-
to-grapheme weight to unity means that lexical
input to grapheme nodes also has a maximum of
1.0.
Dual-route model: Grapheme activation by the
phonological route
Grapheme nodes receive input from the phonolog-
ical route in parallel to the lexical input. The gener-
ation of this input is exactly as described in the
discussion of the phonological route in isolation,
except that it builds up over time. This is achieved
in exactly the same way as for the lexical route, viz. a
maximum input value for each grapheme node is
multiplied by an exponential ramping factor,
+
PhonInput i (t ) = Maxi ∗ α [ ramp − t ]
where PhonInputi(t) is the input to grapheme unit ui
from the phonological route at time t, and Maxi is
the maximum value of this input (cf. equation 7, for
lexical activation). The term Maxi is computed at
the outset and is the activation of a grapheme node i
(prior to grapheme competition) that would be pro-
duced by the phonological route if the input pho-
neme nodes had their maximum activation of 1.0.
Note that the phonological route cannot activate a
grapheme node above a value of 1.0 (i.e., Maxi ≤ 1),
and hence PhonInput, like LexInput, cannot be
greater than 1.0.
138 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
Dual-route model: Summation of inputs and
grapheme competition
The activation of grapheme nodes follows the same
rules as specified earlier for the phonological route
in isolation (equations 2–4), except that input is
(10)
now also received from the lexical route. The acti-
vation GraphAi of a grapheme node is given by,
GraphA i (t ) = δGraphA i (t − 1)
(12)
+ ExtInput i (t ) − IntInput i (t )
where δ (= .6) is a decay parameter; ExtInput (exter-
nal input) is just the sum of the lexical input
(LexInput, equation 10) and the phonological input
(PhonInput, equation 11); and IntInput (internal
input) is the lateral inhibitory input due to other
activated grapheme nodes competing for the same
output position (equation 4). As before, grapheme
node activations are clipped to remain in the range
[0,1].
As discussed above, the input from the two
routes is “ramped,” and the time taken (in simu-
lated time steps) for input to reach its maximum
value depends on the parameter ramp (equations 7
and 11). In principle, this could be set differently
for the two routes, reflecting a greater emphasis on
the use of one route over the other. For instance, a
large difference between the two values would
effectively disable the slower route. In the follow-
ing simulations, it is assumed to be equal for the
two routes (i.e., phonological activation flows into
the two routes at the same rate), and is set to a
value of 10.
(11)
Summary of parameters. The full model with acti-
vation ramping contains more parameters than the
phonological route in isolation (Table 7). They are
set to values that enable the model to produce
robust, unambiguous spelling of all “known”
words (in particular, of low-frequency exception
words) given the fairly strict response criteria spec-
ified above (all grapheme nodes having an activa-
tion of either 1 or 0). The same parameters are
used in all simulations, except in the simulations of
surface dysgraphia, where we model the deficit as a
lowering of the ceiling of activation for lexical
nodes.
 CONNECTIONIST DUAL-ROUTE SPELLING

Table 7. Parameters used in dual-route simulations

Symbol Function Value

Wmax Maximum strength of lexical node excitatory self-feedback. .9

Modulated by lexical frequency.
ramp Number of cycles for phonological input to each route to 10
reach its maximum value. Set to be equal for the two routes.
Wlex (positive) Excitatory lexicon-to-grapheme weight. 1.0
Wlex (negative) Inhibitory lexicon-to-grapheme weight. –.5
w+ Grapheme node self-activation. .6
w- Grapheme node lateral inhibition.
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

–.75

Results: Performance of the dual-route
model
Lexical dominance: Spelling exception words
We showed earlier how the isolated sound-to-
spelling component of the model regularises
inconsistent and exception words on which it has
been trained. However, when used in combination
with the lexical route just described, this should
not cause the model to make large numbers of
errors on these words. That is, when investigating
the full model, it is best to start with a version that
can actually spell irregular words well. This is
achieved by the model as described above, with the
given parameter setting. In this form, the model
correctly spells even low-frequency exception
words (the most “difficult” cases), the lexical route
being able to dominate the combined inputs from
both routes. This dominance is due to a combina-
tion of factors.
First, the input from an activated lexical node to
each grapheme it contains has an asymptotic value
of 1.0. Although the sound–spelling input can also
reach this level for particular graphemes, it fre-
quently does not, and in particular it does not do so
in cases of sound–spelling inconsistency (e.g., /i/ →
EE, EA; see Figure 5). Hence the activation from
the lexicon is stronger where sound–spelling rela-
tionships are inconsistent. Furthermore, if the
word has an inconsistent, rather than truly excep-
tional, spelling then one of the candidates activated
by the phonological route is likely to be the lexically
correct one. In this case the lexical spelling will be
further reinforced. In combination with the lateral
inhibitory interactions, this can suppress lexically-
incorrect phonological activation. All words benefit
from this effect, regardless of frequency.
Second, although the “ramping” parameter is set
to be equal for the two routes (i.e., activation
spreads at the same rate into the two routes), the use
of the frequency-weighted feedback parameter in
the lexical route speeds up lexical activation in pro-
portion to the word frequency. Hence the lexicon-
to-grapheme input can increase more rapidly than
the phonological input, even when both inputs have
the same asymptotic value. This advantage is par-
ticularly enjoyed by higher-frequency words. As
with the first factor, this only causes the suppression
of incorrect graphemes by virtue of the lateral
inhibitory interactions amongst graphemes.
Finally, lexical nodes also send inhibitory input
to graphemes they don’t contain. This feature is
distinct from the first two in not depending on
competition amongst grapheme nodes to have an
effect. In many cases it is fairly redundant, the first
two factors being sufficient to establish lexical
dominance even when lexical input is purely excit-
atory (this is easily achieved in the model imple-
mentation by setting the lexicon-to-grapheme
inhibitory weight to 0). However, it is needed in a
few cases where the phonological route activates a
grapheme in an output position in which the lexical
route activates nothing, in particular a redundant
final-e, e.g., “soon” → SOONE. In this case there
are no competing graphemes to suppress the activa-
tion of the final-e. Ideally, it would be parsimonious
to do without this factor, thus leaving response
competition as the only mechanism for resolving
ambiguous graphemic activation. However, to deal
with cases such as SOONE, this would appear to

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 139

 HOUGHTON AND ZORZI
require the use of “null” grapheme nodes (repre-
senting empty positions): A lexically activated null
grapheme in the last position would compete with,
and suppress, the final-e. However, for consistency
of representation, these null nodes would also have
to be included in the training and output of the
phonological route. This would complicate the
model and may have unexpected side effects. Since
we currently know of no evidence to support the
existence of null grapheme nodes (which can be
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
activated and compete like any other), inhibitory
lexical input is used, though its strength is kept as
low as possible.
It is important here to emphasise a final point:
Although these features result in the dominance of
the lexical route in spelling known words, they do
not remove all influence of the sound-to-spelling
process, and this influence should not be considered
to be merely a source of interference. In particular,
where the two routes are in agreement, the input is
summed, and lexical activation can be strongly rein-
forced by input from the phonological route. Since
most graphemes in the great majority of English
words are in regular (or quasiregular) correspon-
dence to the words’ phonemes, the pooling of the
two sources of graphemic activation is generally
beneficial. That is, phonological input to the nor-
mal lexical-spelling process should not be con-
strued as a source of noise that it would be better to
do without .
Simulation 7: The frequency-by-regularity
interaction
Reading words aloud is known to be affected by the
regularity of the words. Irregular words (those con-
taining at least one low-frequency grapheme-pho-
neme correspondence) have longer reading
latencies than regular words (Baron & Strawson,
1976). This effect has been reported to interact
with word frequency, such that it is particularly
strong for low-frequency words (Seidenberg,
Waters, Barnes, & Tanenhaus, 1984; Taraban &
McClelland, 1987). Related findings have been
reported from acquired surface dyslexic subjects,
who can read nonwords well but tend to regularise
the pronunciation of exception words. The latter
tendency is especially marked for low-frequency
140 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
exception words (McCarthy & Warrington, 1986;
see Zorzi et al., 1998a, for a simulation of surface
dyslexia).
Analogous results have been reported for spell-
ing (Kreiner, 1996; Kreiner & Gough, 1990). For
instance, Kreiner (1996) used an oral spelling test to
examine time taken to spell (polysyllabic) words as a
function of their rated familiarity (a variable corre-
lated with frequency) and degree of regularity. Both
total spelling time and time to start spelling were
measured. Reliable main effects on total spelling
time were found for both familiarity and regularity,
higher values of either factor producing more rapid
spelling. Most importantly, there was a significant
interaction between the factors, the low-familiarity
words being much more affected by regularity than
the high-familiarity words. A similar effect of the
interaction was also found for subjects’ time to start
spelling.
Kreiner (1996, Experiment 3) reports an analo-
gous pattern of results, including the interaction,
when the data were analysed for spelling accuracy.
Low-familiarity words were generally spelled worse
than high-familiarity words, but the effect was
much greater for irregular than for regular words.
Results regarding spelling accuracy in normal sub-
jects are reflected in the impaired spelling of sub-
jects with acquired surface dysgraphia. Recently,
Rapp et al. (2002) have reported data from surface
dysgraphic patient LAT, who achieved 90–98%
correct spelling of pseudowords while showing sig-
nificantly impaired spelling of real words. LAT
showed both a frequency effect (high-frequency
words spelled better than low-frequency words),
and a regularity effect (regular words spelled better
than irregular words). Importantly, Rapp et al.
report a frequency-by-regularity interaction, such
that low-frequency, irregular words were always
spelled with lowest accuracy. This pattern clearly
indicates an impairment in lexical spelling with
spared grapheme–phoneme conversion. This view
is reinforced by the observation that the vast major-
ity (90–100%) of LAT’s spelling errors were
phonologically plausible.
We first examine the model’s behaviour using a
response latency measure. Later we will look at
accuracy in the context of simulating the surface
 CONNECTIONIST DUAL-ROUTE SPELLING

dysgraphic pattern. To examine latency we used

four word lists adapted from the Taraban and
McClelland (1987) study of reading. These lists
(see Appendix A) consist of two types of word stim-
uli (regular and exception), and two frequency
bands (high and low). The same stimuli were used
earlier to confirm the consistency effect in the iso-
lated phoneme–grapheme route. In that simula-
tion, word frequency was irrelevant, and the four
lists were collapsed into two lists of 48 items of
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

mixed frequency. In this simulation, word fre-

quency is a factor and we first wished to ensure that
(for the model) there was no confound between fre-
quency and regularity for the phonological route
(i.e., that the two lists of regular words were equally
regular, the two lists of exception words equally
exceptional—recall that the original word lists were
compiled on spelling-to-sound criteria). Thus we
first presented all four lists (of 24 items each) to the Figure 10. Effects of word frequency and regularity on the time
phonological route in isolation and collected the taken by the dual-route model to produce a consistent, fully
model’s latencies (as described earlier). No effect of activated spelling pattern. Both factors produced significant effects,
frequency was found for either word type, the and the interaction was also significant. The stimulus lists used in
model producing almost identical mean latencies the study are given in Appendix A.

for both lists of regular words and for both lists of

exception words.
Following this, the same word lists were pre-
sented to the full model, and the time taken (in
simulated processing cycles) to produce an unam-
biguous spelling plan was recorded. All the stimuli
were spelled correctly, including the low-frequency
exception words. Figure 10 shows the mean laten-
cies for the four different conditions. The response
latencies were submitted to a two-way ANOVA
with factors Word Type (regular or exception) and
Frequency (low or high). Significant main effects
were found for both factors: Word Type, F(1, 92) =
17.78, p < .001, regular words being faster than
exception words; Frequency, F(1, 92) = 152.22, p <
.001, high-frequency words being faster than low-
frequency words. Importantly, the interaction of
Word Type × Frequency was also significant, F(1,
92) = 10.76, p < .01, that is, change in word fre-
quency has a greater effect on the spelling of excep-
tion words than on regular words. The model thus
shows the basic pattern found in studies of spelling
time for polysyllabic words reported by Kreiner
(1996).
Discussion. The finding of a highly significant fre-
quency effect demonstrates that the lexical fre-
quency manipulation in the model (variation in the
positive feedback weight on lexical nodes) makes
itself felt at the grapheme level, even though
grapheme nodes are also receiving input from a
phonological route that does not show a frequency
effect (indeed, does not show a lexicality effect, as
discussed earlier). Similarly, the consistency effect
found in the phonological route produces an appar-
ent lexical regularity effect—regular words are
spelled significantly faster than irregular ones. The
model would not show this effect without the inter-
active input from the phonological route when
spelling known words. Finally, the frequency-by-
regularity interaction was also significant: The low-
frequency of a word causes particular difficulty if
the word is also irregularly spelled. For these words,
the effect of the relative slowness of the lexical acti-
vation is compounded by (1) the lack of support
from the phonological route for the exceptional PG
mappings in the word, and (2) response competi-

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 141

 HOUGHTON AND ZORZI
tion due to activation of higher frequency
mappings. For high-frequency exception words,
the lexical route activation is sufficiently rapid that
the effects of these two factors is much weaker. In
particular, effects of response competition are much
reduced, as the low-probability graphemes acti-
vated by the lexical route quickly build up a high
activation enabling them to inhibit any activation,
of potentially competing graphemes.
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Modelling impaired spelling: Surface dysgraphia
We showed earlier that the isolated phonological
route could spell regular words and nonwords well,
and consistently regularises exception words. As
such it behaves like a very severe surface dysgraphic
subject. However, such subjects typically show a
preserved ability to spell many exception words,
and in particular exhibit a frequency-by-regularity
interaction such that their spelling of exception
words gets worse as the frequency decreases (e.g.,
patient MP, Behrmann & Bub, 1992; patient
LAT., Rapp et al., 2002). In terms of the current
model, this pattern clearly suggests some general
weakening of the influence of the lexical route
rather than its complete neutralisation. To investi-
gate this possibility we examined the performance
of the full model with the output of the lexical path-
way weakened.
There are a number of manipulations that may
be made to the lexical route which would weaken its
influence on the spelling process. For instance, as
described above, the rate of lexical activation is par-
tially determined by a ramping parameter, which is
set to be equal for both routes in the model. If the
value of this parameter is increased for the lexical
route, then lexical activation will be slowed relative
to the phonological route (see Zorzi et al.,1998a,
for an example of such a manipulation in the con-
text of reading). An alternative proposal is that the
asymptotic strength of the lexical output has been
reduced following neural damage (e.g., due to low-
ered excitability of the neural structures encoding
orthographic lexical knowledge). In this case the
ramping parameter is itself unaffected, but normal
levels of output are never achieved.
In the following studies we use the latter manip-
ulation to simulate the surface dysgraphic deficit.
142 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
The activation ceiling is reduced for all lexical nodes
equally, irrespective of frequency. Hence, to the
extent that the model exhibits the more detailed
surface dysgraphic pattern, it is not due to the rep-
resentation of lower frequency words suffering
greater “damage.” Note that this kind of simulated
lesion can be easily reconciled with the hypothesis
that spelling partly depends upon the integrity of
the semantic system (Graham et al., 2000). Weaker
activation of the OOL might result from the lack of
input from a damaged semantic system, or alterna-
tively from the absence of coherent resonance (via
recurrent feedback) between orthographic and
semantic representations. The detailed modelling
of surface dysgraphia in the context of semantic
impairments (as well as its time course due to the
progressive nature of the deficit) represents an
intriguing challenge but one that is beyond the
scope of the present paper.
Surface dysgraphia: Patient MP. Behrmann and Bub
(1992) present a fairly detailed analysis of the spell-
ing of a surface dysgraphic (and dyslexic) patient,
MP. MP was asked to write to dictation a total of
392 words (mono- and polysyllabic), half of which
were regular and half irregular (see Behrmann &
Bub, 1992, Appendix 3 for a complete list of the
stimuli along with MP’s spelling of them). The
assignment of words to these classes was made on
the basis of spelling-to-sound regularity, i.e., with
respect to reading. Both sets of words were divided
into six frequency bands, ranging from less than 10
per million to more than 200 per million. For spell-
ing, MP showed a significant effect of regularity on
accuracy, spelling regular words better than irregu-
lar words. In addition her spelling of the irregular
items showed a marked frequency effect, accuracy
decreasing with frequency. No equivalent trend of
accuracy against frequency was found for the regu-
lar words, resulting in a frequency-by-regularity
interaction. Although Behrmann and Bub did not
analyse the spelling errors produced by MP any fur-
ther, it is clear from the results provided in their
Appendix 3 that the vast majority of them were
phonologically plausible.
In terms of comparing the model with the data
from MP, a major problem is that the word lists
 CONNECTIONIST DUAL-ROUTE SPELLING

include many polysyllabic words, whereas the

model is restricted to monosyllables. An additional,
less serious, problem results from the use of
grapheme–phoneme correspondences to define
regularity. Consequently some of the supposedly
“regular” words contain rather low-frequency pho-
neme–grapheme correspondences, e.g., siege, niece,
limb, launch, etc. The inclusion of such stimuli in
the regular-word set is likely to contaminate the
results somewhat. Fortunately, Berhrmann and
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

Bub (1992) provide a complete listing of all the

stimuli they used, along with MP’s spelling of
them. We were therefore able to analyse their data
for the monosyllabic words only, and compare this
with the performance of the model. Before looking
at the performance of the model with an impaired
lexical route, we tested the unimpaired model on
the stimuli, and analysed the model’s response
latencies. As well as providing a predicted “normal
profile” for these stimuli, it allows us to examine the
frequency-by-regularity effect reported above in
more detail.
Simulation 8: Normal performance
As noted above, the polysyllabic words had to be
removed from the Behrmann and Bub (1992) stim-
uli, and this occasioned some additional changes. In
particular, the removal of the polysyllabic words led
to there being very few stimuli in some conditions
compared to others (e.g., 7 against 27), so we
supplemented the depleted conditions with more
words of the appropriate frequency and word-type
to get a better sample with which to test the model.
The resulting sets of regular and irregular words are
shown in Appendices, B and C, along with the
mean frequency of each word group. In the end, the
monosyllabic stimuli consisted of 216 words.
The 12 lists were presented to the model and the
response time for each word was recorded. Parame-
ters were exactly as for the previous simulations.
The mean latency for each word list is shown in
Figure 11.
The results were analysed using ANOVA for
two factors, Regularity and Frequency. Both factors
produced significant main effects: Regularity, F(1,
215) = 36, p < .0001; Frequency, F(5, 215) = 22.4, p
< .0001. The Frequency × Regularity interaction
Figure 11. Model latencies for regular and irregular words in six
different frequency bands (for stimuli, see Appendices B and C).
was also significant, F(5, 215) = 2.52, p <.05. This
confirms the results reported above using different
sets of words. It is clear from the graph that the RT
difference between regular and exception words
increases as word frequency decreases, and that at
the highest frequencies there is no regularity effect.
To look at this in more detail, the data were sub-
jected to trend analysis for both factors and the
interaction. The two main effects showed signifi-
cant linear trends, and the only significant trend in
the interaction was also linear (t = 2.68, p <. 01).
Hence the main difference in the two curves is in
their slope, the curve for the exception words being
significantly steeper than that for regular words.
Simulation 9: Weakening the lexical route
Figure 12 shows the spelling accuracy data (per-
centage correct) for MP on the monosyllable set,
for each of the 12 conditions (6 frequency x 2
regularity levels).
It is immediately apparent that the large effect of
regularity reported for the complete set of words
(Behrmann & Bub, 1992) is reproduced when the
monosyllables are analysed in isolation. In addition,
regularity interacts with frequency. As is clearly

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 143

 HOUGHTON AND ZORZI
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Figure 12. Percentage correct spelling of monosyllabic regular and
irregular words by patient MP (data derived from Behrmann &
Bub, 1992, Appendix 3) and by the lesioned model. Scores are
shown for words from six different frequency bands. The model was
fitted to the patient data by finding a value for the strength of the
lexical output that produced the same percentage of correct responses
on the highest-frequency irregular words (74%).
shown by the graph in Figure 12, the effect of fre-
quency is very pronounced for irregular words but
virtually absent for regular words.
In comparing the model to such data, it is
important to be clear on the aims of the simulation.
We wish to demonstrate that a very simple global
impairment to the model provides a good quantita-
tive and qualitative fit to the main features of the
data. Quantitatively, we take these to be a signifi-
cant main effect of regularity, and a frequency-by-
regularity interaction such that relative frequency
has a greater impact on irregular than on regular
words. Qualitatively, the kinds of spelling errors
produced by the model should also resemble those
produced by MP, i.e., be phonologically plausible (a
more stringent test of qualitative similarity is
described below in relation to data produced by
Rapp et al., 2002). It is not an aim of the simulation
8
An additional significant advantage of this anchor point is that this set contains a relatively large sample of monosyllables, 23.
144 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
to obtain a close quantitative fit of the model to
every point in the MP monosyllable data. For one
thing, some of these individual points (in the sub-
ject data) are based on stimulus sets with very few
members (e.g., seven) and may not be very reliable.
In general, fitting detailed data from just a single
subject by searching through a model’s parameter
space is not very revealing, unless the parameters
can be fixed in some independent way. In this case,
we leave the model parameters exactly as described
above for modelling normal performance. The only
change is that the strength of the output of the lexi-
cal pathway is reduced by a constant factor.
The strength of the lexical pathway has to be
chosen to reflect the degree of MP’s deficit in some
reasonable manner. A good “anchor point” should
clearly involve the data most relevant to the descrip-
tion of the condition. Beyond this we suggest that a
“minimal impairment” strategy is preferable; that
is, the impairment to the model should permit it to
match the subject’s best performance (on the rele-
vant condition). This ensures the model is not over-
damaged. The key data in this case involve the
irregular words, and MP performs best on the high-
est frequency ones8. Hence we anchored the model
to the data by finding a value for the lexical strength
which produced an error rate in the model as close
as possible to that shown by MP for the highest-
frequency irregular words. MP scored 74% correct
in this condition. With the lexical strength set to
.34, the model produced the same result. The word
sets for the other 11 conditions were then presented
to the model and the spellings produced were
analysed for accuracy.
The results from the model are shown in Figure
12, together with MP’s performance. As can be
seen, the model results fit the data fairly well. The
regular words show no significant effect of
frequency, while the irregular words show an
advantage for higher-frequency words.
Simulation 10: MP’s spelling errors
As well as reproducing the quantitative error profile
shown above, a model of surface dysgraphia should
 CONNECTIONIST DUAL-ROUTE SPELLING

Table 8. Spellings produced by MP and the model for irregular words from the two
lowest frequency bands

Frequency 0-9 Frequency 10-19

—————————————— —————————————
Target MP Model Target MP Model

womb woom woom tomb tume tom

chrome crome crome ache ake ake
chef shef shef pint pinte 3
yacht yaut yott shove 3 3
awe aw 3 glove 3 3
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

dove 3 3 mild 3 3
hind hinde 3 gauge gaje gage
shone shaun 3 deaf def def
dread 3 3 wool 3 3
sweat swet swet brow 3 3
yearn yurn 3 plough 3 3
sieve sive sive soup 3 soope
soot 3 3 sword sord 3
crook 3 3 dough doe 3
hook 3 3 ton tun tun
brook 3 3 shoe 3 shoo
vow 3 3
sponge spunj spunge
suave swav swarve
cough 3 cof
trough 3 trof
shove shuv 3

3 = correct spelling.

show a good qualitative agreement with the kinds
of errors participants make. In Table 8 we show the
spellings produced by MP and the model for the
irregular words from the two lowest-frequency
bands. As can be seen, there is quite good agree-
ment between the two sets of responses as to which
words are difficult; the model tends to make mis-
takes where MP does, and gets many of the same
words right. The great majority of errors produced
by both model and patient are phonologically
plausible, and in many cases are identical.
Simulation 11: Errors and dual-route
interactions—data from patient LAT
The current model is parallel and interactive. Both
routes receive phonological input in parallel and
interactions occur at the graphemic output stage
where both cooperation and competition take
place. However, it is conceivable that the results
discussed above might be shown by a dual-route
model with neither of these properties. For
instance, it might be proposed (see, e.g., Kreiner,
1992) that access to the lexical route occurs first
(and in isolation), the phonological route only com-
ing into play if no lexical candidate is activated
(hence no parallelism, and no interaction). In such a
scenario, lexical access would be all or nothing, but
probabilistic, with the probability of successful
access being lower for low-frequency words. The
impairment resulting in the surface dysgraphic pro-
file would be due to an overall lowering of the prob-
ability of successful lexical access; access to low-
frequency lexical items would, on average, fail more
often than access to high-frequency items. In the
case of regular words, this would not have much
effect on errors as the phonological route would
generally produce the correct spelling. For irregular
words, the more frequent use of the phonological
route for low-frequency words would lead to the
error profile discussed above.

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 145

 HOUGHTON AND ZORZI

Such an account is quite plausible and, given
the possibility of deliberate, strategic control over
spelling, possibly correct in some cases. Interest-
ingly, though, it makes different predictions than
the current model regarding the occurrence of a
particular class of error. A noninteractive model
will either spell a word lexically (if lexical access is
successful) or phonologically (if it is not), and
hence does not predict the occurrence of individ-
ual spelling errors containing a mixture of ele-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Seeing these results, we examined the errors
produced by the model in simulations of impaired
spelling for evidence of the same effect. Candidate
words would need to contain more than one low-
frequency phoneme–grapheme (PG) mapping.
Rapp et al. (2002) use matched pseudowords to
control for the possibility that the phonological
route sometimes produces low-frequency PG
mappings. This is not necessary in the model, as we
have access to the output produced by the phono-

ments coming from different routes. While we logical route (even if it does not appear in the final
were in the process of preparing this article, we output). Each word can thus serve as its own “con-
became aware of the results of a study by Rapp et trol.” The model was considered to exhibit the
al. (2002) with an acquired surface dysgraphic LAT effect in a given word only if the phonological
patient, LAT. LAT scored 90–98% correct on route was trying to regularise more than one low-
spelling pseudowords. On real words, he spelled probability (lexically correct) mapping, but only
high-frequency words better than low-frequency succeeded on one.
words, and regular words were spelled better than We found a number of examples of the LAT
words containing at least one low-probability effect in the impaired model’s output, and some are
phoneme–grapheme mapping. Low-frequency, listed in Table 9. For each word, the table shows the
irregular words were spelled worst of all. Of LAT’s target spelling (which also represents the lexical
errors, 90–100% were phonologically plausible. input to the grapheme selection process), the pho-
The interesting thing about the errors is that they nological spelling produced by the model (control),
showed clear evidence of integration of both lexi- and the actual spelling due to the combined influ-
cal and sublexical information. This can be shown ence of the two routes. In the actual spelling, under-
to occur when a word contains two (or more) low- lined graphemes originate from the phonological
probability phoneme–grapheme mappings. If the route, while graphemes in italic arise from lexical
phonological route is used alone, then we should input. These results confirm the proposal of Rapp
expect all mappings in the word to be high-proba- et al. (2002) that interactive, dual-route models
bility. What Rapp et al. found was that, in many of should exhibit these effects.
LAT’s phonologically plausible errors, alongside
the regularisation there occurred lexically correct,
low-probability elements. By comparison, his Table 9. Regularisation errors (model) showing lexically correct,
a
spelling of matched pseudowords contained sig- low probability graphemes
nificantly fewer such elements, making it unlikely Phonological
that the sound–spelling process was occasionally Target route Actual
contributing low-probability mappings to the Input word (lexical spelling) spelling spelling
spelling of words. For instance, LAT might spell /tVf/ TOUGH TUFF TUGH
the word “bouquet” (/b@UkeI/) as BOUKET, /skim/ SCHEME SCEAME SCEME
but would spell the matched pseudoword /Sik/ CHIC SHEEK SHI K
/lAf/ LAUGH LARF LARGH
“louquet” (/l@UkeI/) as LOKAY. In this case, in /deIn/ DEIGN DANE DEI NE
“bouquet” only the mapping /k/ → QU has been /tu/ TWO TOO TWOO
regularised, the remaining lexically correct, low- /w0t/ WHAT WOT WAT
probability mappings (e.g., /eI/ → ET) have /lImf/ LYMPH LIMF LY MF
remained. In the case of the matched pseudoword, a
In the actual spelling, underlined graphemes are
all the phoneme–grapheme mappings are regular regularisations from the phonological route, graphemes in
(high probability). italic are low-probability spellings from the lexical route.

146 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)


LAT errors: Other possible mechanisms. This rather
subtle effect provides support for interactivity only
if the model is unable to produce it using either of
the two routes in isolation. In this case, the impair-
ment to the model would have to be different to
that used so far, as reducing or slowing the activa-
tion of either route does not introduce graphemes
into the output (of the isolated slowed route) that
would not otherwise occur. The obvious alternative
would be the addition of random noise to grapheme
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
node activations, a manipulation that has been
used, for instance, in modelling the error patterns
found in graphemic buffer disorder (Glasspool &
Houghton, 2002; Houghton et al., 1994).
Addition of random noise to the output of the
isolated lexical route would not produce the LAT
pattern, because when the lexical route activates the
spelling of, say, lymph it does not produce any acti-
vation of the grapheme F as an alternative to PH
(see Table 9). Although the addition of random
noise to all graphemes might occasionally lead to
the production of an F in the final position, it would
not do so with any greater probability than the pro-
duction of any other grapheme, e.g., G (lymg).
However, as noted above, LAT’s errors are over-
whelmingly phonologically plausible, and hence
this possibility is ruled out.
The possibility of the model showing the effect
due to random noise in the isolated phonological
(assembly) route is rather more interesting. Rapp et
al. (2002) rule this out in the case of LAT by the use
of matched pseudowords as controls, showing that
the production of the (lexically correct) low-proba-
bility graphemes is much more likely to occur for
the word than for the nonword stimuli. If both
types of stimuli were being produced by a noisy
phonological route, one would not expect to find
any such lexicality effect. However, this expectation
depends on the assumption that no plausible
sound-to-spelling conversion mechanism will
show a lexicality effect. This assumption may be
incorrect for models such as this, which are trained
entirely on whole-word stimuli.
Earlier, we showed that if consistency is con-
trolled, the model’s phonological route spells words
and nonwords equally quickly (Simulation 2). In
particular, no difference was found between incon-
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
sistent words and nonwords, indicating that, on
average, they are equivalent in terms of the ambigu-
ity or uncertainty of grapheme activation. Thus,
one would not anticipate that the addition of noise
to grapheme activations should result in a lexicality
effect on the production of low-probability
graphemes. However, given that the latency results
are based on means, it remains a possibility that
some exception words, though regularised, still
produce a higher degree of activation of their
lexically correct, low-probability graphemes than
would matched pseudowords. If this were so, then
addition of noise to the grapheme activations could
result in a higher rate of production of such
graphemes for words than nonwords.
To explicitly test this possibility, we constructed
matched pseudowords (by changing either an
initial or final consonant phoneme) for each of the
eight exception words shown in Table 9. For each
pair, a “critical grapheme” was defined, this being
the lexically correct, low-probability grapheme pro-
duced by the (impaired) dual-route model when
attempting to spell the word (the italic graphemes
in Table 9, column 4). We presented each stimulus
pair (word–pseudoword) to the isolated phonologi-
cal route and compared the initial grapheme activa-
tions (prior to output competition) produced by the
elements of each pair.
First we examined whether, for the word stimu-
lus, the critical grapheme received any activation at
all. If it does not, then it is no more likely to be pro-
duced than any other grapheme, including any of
the phonologically implausible options (of which
there are many more). In this case no further analy-
sis is needed, as the addition of the amount of noise
required to get the critical grapheme produced
would also generate many more phonologically
implausible spellings (which LAT does not do). If
the critical grapheme is activated above baseline,
then the likelihood of its being produced by the
addition of noise depends on its activation level
relative to that of its most active competitor (the
inevitable “winner,” when the model is run without
noise). In this case we compared this difference
with the equivalent one found for the matched
pseudoword. If there is no difference between the
two cases (word vs. pseudoword), then the level of
147
 HOUGHTON AND ZORZI
noise required to produce the low-probability
grapheme for the word stimulus would also cause it
to be produced (with the same frequency) by the
matched pseudoword, contrary to Rapp et al.’s
(2002) findings. The results were unequivocal. For
seven out of the eight words, the phonological route
produced no activation of the critical grapheme.
The one case in which it did was /w0t/ (spelled
WOT, lexically what, critical grapheme A, cf. Sim-
ulation 6). The difference in initial activation
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
between the O and A (O minus A) was .5. For the
matched pseudoword /w0l/ (spelled WOL) the A
was also activated and the activation difference
between O and A was .53, very close to that of the
associated word.
These results demonstrate that it is extremely
unlikely that the phonological route of the model
will show any significant lexicality effect on the
selection of low-probability graphemes. We con-
clude, then, that the Rapp et al. (2002) data do
indeed constitute a stringent test of the interactivity
assumption.
GENERAL DISCUSSION
Many cases of human problem solving exhibit a dis-
tinction between memory-based and analytic or
procedural methods. The memory method recog-
nises previously encountered examples of a prob-
lem, and retrieves the solution from memory (if one
is available). The analytic method applies a general
procedure to the problem, and is especially useful
for novel cases. Dual-route models of spelling and
reading are specific instances of this general
perspective.
While this general view is useful in focusing
attention on commonalties across cognitive
domains, it may not take us very far when it comes
to explaining detailed patterns of data in particular
domains, and many different domain-specific
models are compatible with it. In this paper we have
applied a connectionist version of the dual-route
theory to the problem of spelling. The specific form
of the model emerges naturally as one solution to
the problem of the representational limitations of
the simple linear feedforward model. We refer to
148 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
the architecture embodying this solution as the
dual-route multilayer architecture (DR-ML,
Figure 2).
One route of the DR-ML (the direct route) is
equivalent to the simple two-layer network, and we
have shown that this route will come to act like a
sublexical sound-to-spelling conversion mecha-
nism (a “phonological” route) even when trained
exclusively on whole words. This mechanism regu-
larises irregularly spelled words, while showing
excellent generalisation to nonword spelling. It also
shows a consistency effect in response latencies, and
shows the kind of variability at the level of pho-
neme–grapheme mappings that is characteristic of
the performance of adult spellers of English. As
such it seems to have captured something of the
(implicit) knowledge underlying such performance,
including more subtle aspects such as its context
dependence and degrees of certainty. Some of this
knowledge is functionally equivalent to a simple
verbalisable rule, for instance that the sound /b/ is
written B (Figure 8). However, the kind of graded
contextual influence shown by the model, for
instance in vowel spelling, is less easily expressible
in the form of explicit rules. The great attraction of
the connectionist framework is that both types of
relationship can be represented (and learned) in the
same way, as sets of associative weights. The postu-
lation of a competitive output mechanism allows
the model to represent inconsistent sound–spelling
mappings; the model generates multiple possible
spellings in parallel, but eventually settles on the
one that is, overall, most consistent with the sound
of a word.
In the second route of the model, interactions
between sound and spelling are mediated by a
further representational level (or levels). In
connectionist terms, this translates into at least
one additional layer of units lying between the
input and output layers. Because of the generalisa-
tion ability of the direct route of the model, it is
possible for the mediated pathway to operate as a
localist lexical route (of the type postulated
in many models of speech recognition, e.g.,
McClelland & Elman, 1986). In this paper, we
have assumed that this is the case. Input of a famil-
iar phonological word leads to the activation of an

orthographic lexical node, which in turn activates
an associated spelling pattern. We also made the
simplifying (and possibly false) assumption that all
known words are equally well learned, in the sense
that they are equally capable (in principle) of acti-
vating the correct orthographic pattern. Lower-
frequency words, however, become activated more
slowly due to their having a relatively weak ability
to support their own activation, resulting in slower
activation of grapheme nodes. When the two
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
routes are activated in parallel, this results in a
greater influence of the phonological route on the
spelling of low-frequency words, though it is still
possible for the lexical route to dominate output
where the two routes disagree. Nevertheless, the
interaction of the two routes produces important
results for spelling of known words in terms of
response latencies: There is a main effect of regu-
larity, and a significant interaction of frequency
with regularity-consistency.
Further effects of the interaction of the two
routes only emerge when the lexical route is subject
to a lesion in the form of a reduced ceiling of activa-
tion. Such a lesion produces a good quantitative and
qualitative simulation of the spelling of two surface
dysgraphic patients. In particular the model shows
an effect that has only recently been reported (Rapp
et al., 2002) and that we would not otherwise have
thought to look for. As far as we are aware, this is
the first time that such a range of data from
dysgraphic subjects has been successfully simulated
by a computational model.
Relationship of the model to other models
of spelling
A number of connectionist models of spelling have
been produced previously, mainly inspired by the
Seidenberg and McClelland (1989) model of read-
ing, which was most notable for the challenge it
threw down to the traditional dual-route frame-
work. In this model (and its successor, Plaut et al.,
1996), a single (nonsemantic) route translates
between the spelling and the pronunciation of a
word or word-like stimulus. The route consists of a
three-layer, feedforward connectionist network
(trained using backpropagation), with hidden-
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
units and without direct connections between the
input and output units (Figure 2b). Such a model
can produce distributed representations at the hid-
den level, which allows it to generalise what it has
learned to novel stimuli (pseudowords). Thus the
functions performed by the traditional lexical and
sublexical reading processes are intertwined in a
single mechanism.
Brown and Loosemore (1994) describe a
connectionist spelling model very similar to the
Seidenberg and McClelland (1989) reading model,
including the use of highly distributed input
(sound) and output (spelling) representations. The
model was trained on a small vocabulary of 227
words, the focus of the study being on the rate at
which words were learned as a function of their
sound–spelling regularity. All results reported from
the model are in terms of its “error score” alone. The
model does not produce differences in response
latency, and no examples are given of its actual
outputs.
It was found that regular-consistent words were
learned faster than both irregular words and
“unusual” words (ones with unique rimes, such as
bulb). Similar findings are reported from a study
with children in four different age groups. Brown
and Loosemore (1994) also produced developmen-
tally “dyslexic” versions of their model by reducing
the number of hidden units. These models learned
more slowly than the normal model, but still
showed the regularity effect. Though no detailed
analysis of the normal model’s nonword spelling is
provided, when compared to the dyslexic versions it
performed better (even when level of regular word
spelling was held constant across models). How-
ever, judging by Brown and Loosemore’s Figure
16.5, the nonword spelling of their normal model
was actually rather poor; its error score on
nonwords was more than twice that produced on
regular words, and rather higher than that pro-
duced by their “severely dyslexic” model on irregu-
lar words (Brown & Loosemore, 1994, Figure
16.4a). Thus, although the model showed an
advantage for regular word spelling, there is no
indication that it abstracted the kind of representa-
tion necessary to support good spelling on
nonwords. It would therefore not be expected to
149
 HOUGHTON AND ZORZI
exhibit the surface dysgraphic pattern if damaged in
some way.
The spelling model of Olson and Caramazza
(1994) was also inspired by work on reading, in this
case the Sejnowski and Rosenberg (1987) NETtalk
model. Like the Brown and Loosemore (1994)
model, this is a three-layer feedforward network,
trained with the backpropagation algorithm. The
model functions serially: The phonemic input is
presented by a “moving window”, centered on the
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
current phoneme, which moves from “left to right”
as the word is presented. This allows the model to
“see” each phoneme in the word in sequence, along
with preceding and upcoming phonemic context.
At the output level, a single grapheme is activated at
each time step, intended to correspond to the pho-
neme in the central position of the window. Note
that, in contrast to the model presented in this
paper, this method provides explicit training on
phoneme–grapheme relationships, as the model’s
“attention” is centred on one phoneme at a time,
and the corresponding output grapheme (of the
current word) is simultaneously presented in isola-
tion. On this basis, one would anticipate that the
Olson and Caramazza model should perform better
than the Brown and Loosemore model on nonword
spelling. As with the latter model, Olson and
Caramazza assess their model exclusively by the
errors it makes. It is not clear how the architecture
could produce variation in response latency.
The model was trained on corpora of 1000 and
1628 words, depending on the simulation. Both
corpora contained both regular and irregular words.
The best performance on the training set was
reported for the 1628-word model. After 100
epochs it spelled 83% of the words correctly, at
which point the error curve appeared to be at
asymptote. The authors report various attempts to
improve on this level of performance, but they were
unsuccessful. Of some importance, then, are the
errors the model made, in particular whether they
were phonologically plausible. Olson and
Caramazza (1994) report that the majority of cases
were actually omission errors, defined as the model
failing to activate any grapheme above a threshold.
Such errors would result in phonologically implau-
sible spellings. However, of the nonomission
150 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
errors, most were phonologically plausible. On a
generalisation test using words the model had not
been trained on, the 1628-word model performed
quite well, producing 87% plausible spellings.
When damaged by the addition of noise to the
weights, the number of errors increased and irregu-
lar words were more affected than regular words,
but again with many implausible errors. This rather
weak regularity effect was confirmed when the
damaged model was required to spell untrained
words. For a mild lesion, plausible spellings of
untrained words fell to around 70%, and with a
stronger lesion it fell to about 50%. Olson and
Caramazza conclude that, overall, the behaviour of
the model provides only a weak approximation to
the surface dysgraphic pattern; in both its normal
and damaged state, it produces too many implausi-
ble spellings to be considered a viable model of sub-
ject data.
A similar model, again using the sliding window
approach, has been proposed by Bullinaria (1994,
1997). This model was trained on a corpus of 2837
monosyllabic words, presented on the basis of their
log frequency, and achieved 100% correct spelling
of the corpus after 700 epochs. Like the Brown and
Loosemore (1994) model, it learned regular words
more easily than exception words. The generalisa-
tion performance of the model (nonword spelling)
was 88.6% correct, very similar to that achieved by
the Olson and Caramazza (1994) model. As with
the other models discussed in this section,
Bullinaria’s model only produces variation in error
scores, not in response latencies. However,
Bullinaria (1994) follows Seidenberg and
McClelland (1989) in maintaining that error scores
in the model should be monotonically related to
human response times. On this basis, the model
shows a regularity effect (regular vs. exception
words), but no frequency effect and no frequency by
regularity interaction. Damage to the model (in the
form of weight scaling) has the greatest effect on
the model’s spelling of exception words, a basic
element of the surface dysgraphia. Additional
features of surface dysgraphia are not simulated,
and the possibility of also observing phonological
dysgraphia in the same model is ruled out
(Bullinaria, 1994). Bullinaria concludes that the

model suffers from the lack of any kind of lexical
route, but that it might serve as a model of the
sound-to-spelling route. However, if this were so
then it would be a sound-to-spelling model capable
(in principle) of learning to spell exception words,
which our model cannot do.
In conclusion, previous models of spelling were
inspired by the single-route reading models of the
1980s. As might be expected, they suffer from
many of the problems these latter models have run
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
into. No such model has so far proved able to simu-
late the range of data considered in this paper; this is
true both of the detailed features and also of a class
of data, variation in response latency. None of the
models discussed above have any mechanism
whereby this can vary.
Limitations of the model
The present model is limited in a number of ways.
Below we discuss three areas clearly meriting fur-
ther investigation: learning; representation of poly-
syllabic words; and output processes, in particular
control of serial order.
Learning. The phonological route of the model is
trained, and extracts its sound-to-spelling knowl-
edge from what we consider to be a reasonable
sample of English words (all the uninflected mono-
syllables). The corpus is not “filtered” in any way to
bias results in a desired direction, and is of the kind
that has been used in other spelling models, and in
related work on reading, permitting models to be
compared on a consistent basis. However, we
would not advance the model as it stands as a model
of learning to spell.
One reason is related to this corpus. Children do
not learn to read and write from a few exposures
(about 20 in the simulations) to the whole set of
monosyllabic words of English (many of which are
of very low-frequency). Rather they begin with a
smaller set, which they see many times, and many of
which are polysyllabic and /or inflected. A realistic
learning model should clearly reflect more closely
the stimulus environment of the child learning to
spell. That said, judging by the results of the
simulations, the model actually does a good job of
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
extracting and representing the sound-to-spelling
regularities of English. This is based on the statisti-
cal regularities present in its training corpus. Any
corpus with the same properties would lead to the
same result. Thus to the extent that regularities
present in the set of English monosyllables are rep-
resentative of those in the language more generally,
the eventual outcome should be the same. How-
ever, the details of the developmental progression
might well be affected if the sound-to-spelling rela-
tionships of the set of words children are exposed to
are not statistically stationary.
A more important limitation to the model is that
the lexical route is at present not learned. This
affects the development of the phonological route,
as it learns with no contribution from a lexical route.
If the latter were developing in parallel, then it
would be expected to contribute by reducing the
error, and this could have a knock-on effect on what
the phonological route learns. One possibility is
that the lexical route might contribute most to what
it is most important for, i.e., spelling irregular and
inconsistent words. These possibilities were inves-
tigated by Zorzi et al. (1998a) in their reading
model (which has the same architecture as this
model). The model was trained on the monosyl-
labic word set (from spelling to sound), with learn-
ing taking place in both direct and mediated
(hidden unit) pathways at the same time. This ver-
sion of the model was capable of learning the whole
training set, including the exception words. How-
ever, it was found that the direct route (when stud-
ied in isolation) still behaved like a spelling-to-
sound conversion mechanism, and did not acquire
lexical properties. The hidden unit pathway
behaved more like a (distributed) lexical route.
When the model was trained with relatively few
hidden units (restricting its capacity to represent
the training set), they appeared to dedicate them-
selves to the exception words, by “correcting” the
output of the direct route, which, left to itself,
would regularise them. Hence, there is no problem
in principle for the DR-ML architecture to learn in
both routes simultaneously, and there is evidence
that it will tend to “self-modularise” so that the
regular, productive, sound-spelling mapping will
be carried out by the direct route (the two-layer
151
 HOUGHTON AND ZORZI
phonological route of the current model). Notably,
a very similar result has also been obtained in the
context of learning the past tenses of English verbs
(Thomas & Karmiloff-Smith, in press).
In the simulations of Zorzi et al. (1998a, 1998b),
backpropagation was used to train the hidden-unit
route. This algorithm tends to generate distributed
(i.e., non-orthogonal) representations at the hid-
den units. The current model, however, simulates
the lexical route as a localist network, in which each
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
known word is orthogonally represented. Such rep-
resentations can be formed in neural networks
using, for instance, competitive learning algorithms
(Grossberg, 1980; Kohonen, 1984). Our proposal
that frequency affects the strength of the feedback
weight, by which a representation supports its own
activation, is compatible with competitive learning;
if each time a node is activated, its feedback loop is
strengthened (thus enabling it to fare better in the
competition for activation, which is essential to
such algorithms), then more frequent words will be
more easily activated. Thus there should be no dif-
ficulty in principle with adding learning to the lexi-
cal route.
A further important learning issue involves the
model’s orthographic representation. The model is
supplied at the outset with a syllabically aligned
graphemic representation, which is of great benefit
to its learning of the sound–spelling mapping.
While it is known that children have developed syl-
labic phonology by the time they start to learn to
read and write (Treiman, 1994), their orthographic
representations must develop as a part of this learn-
ing. Hence a plausible developmental model of
spelling cannot start out with complex grapheme
nodes, and orthographic syllable structure. In some
cases children may acquire graphemes by being
taught them directly, e.g., TH, SH, CH, etc. As a
result these short sequences become chunked, and
available for use in representing longer sequences.
However, not all orthographic knowledge of this
sort is directly taught. Treiman (1993) provides an
interesting example of the grapheme CK, which
cannot occur syllable-initially in English (though
its associated phoneme /k/ can). Treiman found
that although first-graders sometimes used this
grapheme incorrectly (for instance, spelling bike as
152 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
BICK), they hardly ever used it at the beginning of
a word, only in the middle and at the ends. Most
importantly, this positional restriction (and others
like it) was not explicitly taught in the school.
Hence even children at the beginning of formal
instruction in spelling appear to have (implicitly)
extracted some representation of the positional
availability of graphemes from the structure of the
words they see.
The present model ought to be able to accom-
modate such findings. It never produces CK at the
beginning of a word (even early in learning) because
it encodes grapheme identity along with position.
Since CK never appears in the word initial position,
it never co-occurs with an initial /k/. A related
developmental account might go as follows.
Children first need to develop representations of
individual letters for the purpose of writing them.
This generally occurs before learning to spell whole
words (Treiman, 1994), and the representations
formed are consequently independent of letter
position. However, with the experience of writing
words, it is obvious that letter position must also be
encoded in some form (rat, art, and tar are not the
same word). With increasing experience, conjunc-
tive representations may develop of “graphemes in
positions,” but only for those combinations that are
actually encountered. On this account, children’s
sensitivity to positional constraints would reflect
their ability to produce representations containing
an element of serial position.
Representation of polysyllabic words. The model can
only handle monosyllabic words. How could it be
extended to deal with polysyllabic words? One
obvious problem that might arise concerns the
model’s positional orthographic representation.
The model proposes that (good) spellers have
developed conjunctive representations of
graphemes in syllabic positions. If literally extended
to longer words, this would mean that separate
nodes would be needed for, say, all the initial conso-
nants in the 2nd syllable of a word, all the vowels in
the 2nd syllable, all final consonants in the 3rd
syllable, and so on. Intuitively this is not a very
appealing prospect, though given that words are of
finite length it is not actually an impossibility.

However, one serious problem would be that learn-
ing of the sound-to-spelling relationship for one-
syllable words would not generalise to polysyllabic
words, because they would use a different set of
position-specific grapheme nodes. Knowing that
an initial /b/ is spelled B in monosyllabic words
would not help the learner spell the last syllable of
Beelzebub.
One solution to this problem is to emphasise the
serial nature of spelling, and propose that it pro-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
ceeds on a syllable-by-syllable basis. Thus in the
orthographic lexicon, polysyllabic words would be
represented hierarchically as a sequence of syllables.
Each syllable node would be connected to the
graphemes in that particular syllable, and a single
set of grapheme nodes could be used to represent all
syllables. Spelling the word would involve the sylla-
ble nodes becoming active in sequence. For the
phonological route the principle would be the same.
The phonological form of polysyllabic words (and
nonwords) would be presented one syllable at a
time, and thus would be spelled by iterating over the
process of spelling monosyllabic words.
Hence we propose that spelling polysyllabic
words in this model requires the addition of more
hierarchical representations and of serial processes.
However, even spelling a monosyllabic word is
serial at the level of the individual letters, but our
model stops at the parallel activation of a set of
grapheme nodes representing a spelling “plan.”
This raises the issue of how this plan is actually real-
ised as a sequence of actions.
Serial order and the graphemic output buffer. Some of
the models discussed above (Bullinaria, 1994;
Olson & Caramazza, 1994) present the phonologi-
cal input serially using the sliding window tech-
nique. This leads directly to serial activation of
grapheme nodes at the output. If we assume that
the activation of each output node is translated into
a response as it becomes active (i.e., no “buffering”
takes place), then these models do not generate a
spelling plan in the way our model does. The
advantage of this is that, for spelling, there is no
“problem of serial order”—the serial order is in the
phonology. An apparent prediction of such a model
is that all spelling must therefore be via phonology;
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
in particular, there can be no direct access to the
spelling of a word from its meaning (as it contains
no serial order information). Most current func-
tional models of spelling reject this position, postu-
lating the existence of a direct route from semantics
to the orthographic output lexicon (Figure 1). This
view is supported, for instance, by studies of sub-
jects who show better written than oral naming
(Bub & Kertesz, 1982). On this account, it is
unlikely that the processes controlling serial output
in spelling are simple reflexes of those of the speech
system.
The idea that sequentially ordered behaviour
involves a stage of parallel activation of a set of
responses has a long history, and indeed was central
to Lashley’s (1951/1960) influential arguments
against associative chaining (see Houghton &
Hartley, 1996, for discussion). Box-and-arrow
models of spelling endorse this view by postulating
the existence of a “graphemic output buffer” in
which a number of responses are activated before
being executed (Morton, 1980; Shallice, 1988).
Our model is clearly compatible with this idea. A
specific acquired disorder of the graphemic buffer
has been identified (Caramazza, Miceli, Villa, &
Romani, 1987) and studied in a number of patients
(see Shallice et al., 1995, for a review). The impair-
ment mainly involves errors in letter selection and
ordering, and is similar for both words and
nonwords. Two models of the graphemic buffer
and disorders of it have been proposed, the
symbolic Multiple Object Spelling (MOS) model
of Caramazza and Miceli (1990), and the
connectionist Competitive Queuing (CQ) model
of Houghton, Glasspool, and Shallice (1994; see
also Glasspool & Houghton, 2002). Detailed
discussion of these models and the data they are
intended to account for is beyond the scope of this
article. However, it is important to consider how
compatible they are with the proposals made in the
current model.
The MOS model is in essence a model of spell-
ing representation (Shallice et al., 1995), and has
little to say about serial output processes per se. It
proposes that spelling plans in the graphemic buffer
include information regarding orthographic sylla-
ble structure, the consonant–vowel status of indi-
153
 HOUGHTON AND ZORZI
vidual letters, and whether a letter is to be doubled.
Our model is broadly consistent with these propos-
als. The main difference appears to be that our
model contains complex grapheme nodes, whereas
it is not clear that these are included in the MOS
model. In all descriptions we have seen of the model
(which has not been implemented), syllable nodes
are decomposed directly into individual letters (see,
e.g., Link & Caramazza, 1994). The importance of
complex graphemes in spelling output representa-
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
tions may therefore be a distinct prediction of the
present model.
The CQ model of Houghton and colleagues
(1994) was developed to simulate the serial produc-
tion of letters. When a word is to be spelled, a group
of letter nodes are activated in parallel, but with a
gradient of activation over them such that letters are
more active the sooner they are to be produced. Let-
ters compete to be produced depending on their
activation level. Under disruption (in the form of
noise added to letter activations) the model has
proved capable of accounting for many basic fea-
tures of the errors found in graphemic buffer disor-
der, such as word length effects, the serial position
curve, and error types (Shallice et al., 1995). In the
first version of the model, a word was represented
simply as a series of letters, with no further structure
other than a special marking of doubled letters
(geminates). A more recent version by Glasspool
and Houghton (2002) includes a representation of
the consonant–vowel status of letters, but like the
MOS model, does not use complex grapheme
nodes.
The CQ model is compatible with the current
model if it is assumed that, following the initial par-
allel activation of a set of grapheme nodes proposed
here, there is a process that converts this activation
into a gradient of activation over a set of letter
nodes. The first stage of activation would not only
provide information regarding consonant-vowel
status of letters (as required by the model of
Glasspool & Houghton, 2002), but also informa-
tion about the position of letters in the graphemic
syllable, as required by Caramazza and colleagues’
MOS model. The picture that emerges is therefore
of a two-stage output process, consisting of an ini-
tial parallel stage (including the activation of com-
154 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
plex grapheme nodes) followed by the “unpacking”
of this spelling plan into the activation of a series of
individual letters to control the serial behaviour of
writing (or typing, or oral spelling). This final out-
put stage would have the dynamic characteristics
proposed by CQ models, modulated by informa-
tion regarding “graphosyllabic” structure, as pro-
posed by Caramazza and colleagues, and as
implemented in the current model.
Although the two-stage idea may seem some-
what complex, it is inevitable in any model that
activates a group of grapheme nodes (more gener-
ally “response nodes”) in parallel, i.e., any model
with the functional equivalent of a response buffer.
Hence, such a two-stage process is also required by
many current models of speech production (Dell,
Schwartz, Martin, Saffran, & Gagnon, 1997; Rapp
& Goldrick, 2000) in which activation of the pho-
nemes of a syllable occurs in parallel. Clearly, some
additional process must convert this parallel activa-
tion into the serial order of speech sounds (see
Hartley & Houghton, 1996). This point applies
also to the output stage of current models of reading
aloud, which stop at the parallel activation of a set of
phonemes in a syllabic frame (Plaut et al., 1996;
Zorzi et al., 1998a).
Conclusions
In this paper we have presented the first fully
implemented dual-route model of spelling. We
have argued that this architecture can be derived as
one solution to the problem of the computational
limitations of the simplest connectionist feed-
forward network. It differs from the more
conventional single-route multilayer network in
maintaining the direct connections between the
input and output units. This permits the network
to partition its knowledge to some extent. In par-
ticular we have shown that the direct route, when
attempting to learn to spell a representative sample
of English words, ends up behaving like a
sublexical sound-to-spelling conversion route.
This route in isolation shows a consistency effect
on response latency but no lexicality effect. The
spellings it produces are virtually always phono-
logically plausible, and the variation and context-

sensitivity it shows matches well with the available
data from adult subjects, both normal and
impaired. When combined with a frequency-sen-
sitive lexical route, the resulting interactive model
shows a frequency-by-regularity interaction on
response latencies for spelling words. Weakening
the lexical route permits a more detailed qualita-
tive and quantitative simulation of the surface
dysgraphic syndrome than has previously been
achieved. This includes the finding that the model
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
produces spelling errors containing blends of
(low-probability) lexical and nonlexical elements,
an aspect of surface dysgraphia that has only
recently been explored (Rapp et al., 2002).
In conclusion, the model provides strong sup-
port for a dual-route architecture for spelling, in
which there is parallel access to the two routes and
interaction between them at the level at which a
spelling plan is formed. The sound-to-spelling
route need not be specified as such a priori, but will
emerge from the attempt to predict spelling from
sound, as long as the two levels of representation
can make direct contact with each other. Implicit
knowledge of the sound-to-spelling relationship in
English is best characterised as associative, and
based on the extraction of statistical relationships of
varying degrees of consistency.
Manuscript received 16 October 2001
Revised manuscript received 28 October 2002
Revised manuscript accepted 14 November 2002
REFERENCES
Baron, J., & Strawson, C. (1976). Use of orthographic
and word-specific knowledge in reading words aloud.
Journal of Experimental Psychology: Human Perception
and Performance, 2, 386–392.
Barry, C. (1994). Spelling routes (or roots or rutes). In G.
D. A. Brown & N. C. Ellis (Eds.), Handbook of
spelling: Theory, process and intervention (pp. 27–49).
Chichester, UK: John Wiley.
Barry, C., & Seymour, P. H. K. (1988). Lexical priming
and sound-to-spelling contingency effects in
nonword spelling. Quarterly Journal of Experimental
Psychology, 40A, 5–40.
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
Baxter, D. M., & Warrington, E. K. (1987). Ideational
agraphia: A single case study. Journal of Neurology,
Neurosurgery, and Psychiatry, 49, 369–374.
Beauvois, M. F., & Derousné, J. (1981). Lexical or ortho-
graphic agraphia. Brain, 104, 21–49.
Behrmann, M. (1987). The rites of righting writing:
Homophone remediation in acquired dysgraphia.
Cognitive Neuropsychology, 4, 365–384.
Behrmann, M., & Bub, D. (1992). Surface dylexia and
dysgraphia: Dual routes, single lexicon. Cognitive
Neuropsychology, 9, 209–251.
Brown, G. D. A. (1987). Resolving inconsistencies: A
computational model of word naming. Journal of
Memory and Language, 26, 1–23.
Brown, G. D. A., & Ellis, N. C. (Eds.). (1994). Hand-
book of spelling: Theory, process and intervention.
Chichester, UK: John Wiley.
Brown, G. D. A., & Loosemore, R. P. W. (1994),
Computational approaches to normal and impaired
spelling. In G. D. A. Brown & N. C. Ellis (Eds.),
Handbook of spelling: Theory, process and intervention.
(pp. 319–336). Chichester, UK. John Wiley.
Bub, D., Cancelliere, A., & Kertesz, A. (1985). Whole-
word and analytic translation of spelling to sound in a
non-semantic reader. In K. E. Patterson, J. C.
Marshall, & M. Coltheart (Eds.), Surface dyslexia:
Neuropsychological and cognitive studies of phonological
reading (pp. 15–34). Hove, UK: Lawrence Erlbaum
Associates Ltd.
Bub, D., & Kertesz, A. (1982). Evidence for lexico-
graphic processing in a patient with preserved writ-
ten over oral single word naming. Brain, 105, 697–
717.
Bullinaria, J. (1994). Connectionist modelling of spell-
ing. In Proceedings of the Sixteenth Annual Conference of
the Cognitive Science Society (pp. 78–83). Hillsdale,
NJ: Lawrence Erlbaum Associates Inc.
Bullinaria, J. (1997). Modelling reading, spelling and
past tense learning with artificial neural networks.
Brain and Language, 59, 236–266.
Bullinaria, J., & Chater, N. (1995). Connectionist
modelling: Implications for cognitive neuro-
psychology. Language and Cognitive Processes, 10,
227–264.
Caramazza, A., & Miceli, G. (1990). The structure of
graphemic representations. Cognition, 37, 243–297.
Caramazza, A., Miceli, G., Villa, G., & Romani, C.
(1987). The role of the graphemic buffer in spelling:
evidence from a case of acquired dysgraphia. Cogni-
tion, 26, 59–85.
155
 HOUGHTON AND ZORZI
Castles, A., & Coltheart, M. (1996). Cognitive correlates
of developmental surface dyslexia: A single case study.
Cognitive Neuropsychology, 13, 25–50.
Cipolotti, L., & Warrington, E. K. (1995). Semantic
memory and reading abilities: A case report. Journal of
the International Neuropsychological Society, 1, 104–
110.
Coltheart, M. (1978). Lexical access in simple reading
tasks. In G. Underwood (Ed.), Strategies of informa-
tion processing. London: Academic Press.
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Coltheart, M. (1985). Cognitive neuropsychology and
the study of reading. In M. I. Posner & O. S. M.
Marin (Eds.), Attention and performance XI (pp. 3–
37). Hillsdale, NJ: Lawrence Erlbaum Associates Inc.
Coltheart, M., & Funnell, E. (1987). Reading and
writing: One lexicon or two? In A. Allport, D. G.
MacKay et al. (Eds), Language perception and produc-
tion: Relationships between listening, speaking, reading
and writing (pp. 313–339). London: Academic Press.
Coltheart, M., Masterson, J., Byng, S., Prior, M., &
Riddoch, J. (1983). Surface dyslexia. Quarterly Journal
of Experimental Psychology, 37A, 469–495.
Coltheart, M., & Rastle, K. (1994). Serial processing in
reading aloud: Evidence for dual-route models of
reading. Journal of Experimental Psychology: Human
Perception and Performance, 20, 1197–1211.
Coltheart, M., Rastle, C., Perry, C., Langdon, R., &
Ziegler, J. (2001). DRC: A dual route cascaded model
of visual word recognition and reading aloud. Psycho-
logical Review, 108, 204–258.
Dell, G. S., Schwartz, M. F., Martin, N., Saffran, E. M.,
& Gagnon, D. A. (1997). Lexical access in aphasic
and nonaphasic speakers. Psychological Review, 104,
801–838.
Demonet, J. F., Chollet, F., Ramsay, S., Cardebat, D.,
Nespoulous, J. L., Wise, R. et al. (1992). The anat-
omy of phonological and semantic processing in
normal subjects. Brain, 115, 1753–1768.
Denes, F., Cipolotti, L., & Zorzi, M. (1999). Acquired
dyslexias and dysgraphias. In G. Denes & L.
Pizzamiglio (Eds.), Handbook of clinical and experi-
mental neuropsychology (pp. 289–317). Hove, UK:
Psychology Press.
De Partz, M.-P., Seron, X., & Van der Linden, M.
(1992). Re-education of a surface dysgraphia with a
visual imagery strategy. Cognitive Neuropsychology, 9,
369–401.
Ellis, A. (1988). Normal writing processes and peripheral
acquired dysgraphias. Language and Cognitive
Processes, 3, 99–127.
156 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
Forster, K. I., & Chambers, S. (1973). Lexical access and
naming time. Journal of Verbal Learning and Verbal
Behaviour, 12, 627–635.
Frith, U. (Ed.). (1980). Cognitive processes in spelling.
London: Academic Press.
Funnell, E. (1996). Response biases in oral reading: An
account of the co-occurrence of surface dyslexia and
semantic dementia. Quarterly Journal of Experimental
Psychology, 49A, 417–446.
Gelb, I. J. (1952). A study of writing. Chicago: University
of Chicago Press.
Glasspool, D., & Houghton, G. (2002). Serial order and
consonant–vowel structure in a model of disordered spell-
ing. Manuscript submitted for publication.
Glosser, G., Grugan, P., & Friedman, R. B. (1999a).
Comparison of reading and spelling in patients with
probable Alzheimer’s disease. Neuropsychology, 13,
350–358.
Glosser, G., Kohn, S. E., Sands L., Grugan, P. K., &
Friedman, R. B. (1999b). Impaired spelling in
Alzheimer’s disease: A linguistic deficit? Neuro-
psychologia, 37, 807–815.
Gluck, M. A., & Bower, G. H. (1988a). Evaluating an
adaptive network model of human learning. Journal of
Memory and Language, 27, 166–195.
Gluck, M. A., & Bower, G. H. (1988b). From condi-
tioning to category learning: An adaptive network
model. Journal of Experimental Psychology: General,
117, 227–247.
Goodman, R. A., & Caramazza, A. (1986). Aspects of
the spelling process: Evidence from a case of acquired
dysgraphia. Language and Cognitive Processes, 1, 263–
296.
Goodman-Schulman, R., & Caramazza, A. (1987).
Patterns of dysgraphia and the nonlexical spelling
process. Cortex, 23, 143–148.
Goswami, U., & Bryant, P. (1990). Phonological skills and
learning to read. Hove, UK: Lawrence Erlbaum Asso-
ciates Ltd.
Graham, N. L. (2000). Dysgraphia in dementia.
Neurocase, 6, 365–376.
Graham, N. L., Hodges, J. R., & Patterson, K. (1994).
The relationship between comprehension and oral
reading in progressive fluent aphasia. Neuro-
psychologia, 32, 299–316.
Graham, N. L., Patterson, K., & Hodges, J. R. (1997).
Progressive dysgraphia: Co-occurrence of central and
peripheral impairments. Cognitive Neuropsychology,
14, 975–1005.

Graham, N. L., Patterson K., & Hodges, J. R. (2000).
The impact of semantic memory impairment on
spelling: Evidence from semantic dementia.
Neuropsychologia, 38, 143–163.
Grainger, J., & Jacobs, A. M. (1996). Orthographic
processing in visual word recognition: A multiple
read-out model. Psychological Review, 103, 518–565.
Grossberg, S. (1980). How does a brain build a cognitive
code? Psychological Review, 87, 1–51.
Hahn, U., & Nakisa, R. C. (2000). German inflection:
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Single route or dual route? Cognitive Psychology, 41,
313–360.
Hall, D. A., & Riddoch, M. J. (1997). Word meaning
deafness: Spelling words that are not understood.
Cognitive Neuropsychology, 14, 1131–1164.
Hanley, R., Hastie, K., & Kay, J. (1992). Developmental
surface dyslexia and dysgraphia: An orthographic
processing impairment. Quarterly Journal of Experi-
mental Psychology, 44A, 285–319.
Harm, M., & Seidenberg, M. S. (1999). Phonology,
reading acquisition, and dyslexia: Insights from
connectionist models. Psychological Review, 106, 491–
528.
Hartley, T., & Houghton, G. (1996). A linguistically-
constrained model of short-term memory for
nonwords. Journal of Memory and Language, 35, 1–31.
Hatfield, F. M., & Patterson, K. E. (1983). Phonological
spelling. Quarterly Journal of Experimental Psychology,
35A, 451–468.
Hinton, G. E., & Shallice, T. (1989). Lesioning an
attractor network: Investigations of acquired dyslexia.
Psychological Review, 98, 74–95.
Houghton, G., Glasspool, D., & Shallice, T. (1994).
Spelling and serial recall: Insights from a competi-
tive queuing model. In G. D. A. Brown & N. C. Ellis
(Eds.), Handbook of spelling: Theory, process and inter-
vention (pp. 365–404). Chichester, UK: John Wiley.
Houghton, G., & Hartley, T. (1996). Parallel models of
serial behaviour: Lashley revisted. Psyche, 2. Retrieved
month, day, year, from, http://psyche.cs.monash.edu.
au/v2/psyche-2-25-houghton. html
Houghton, G., & Zorzi, M. (1998). A model of the
sound–spelling mapping in English and its role in
word and nonword spelling. Proceedings of the Twenti-
eth Annual Conference of the Cognitive Science Society
(pp. 490–495). Mahwah, NJ: Lawrence Erlbaum
Associates Inc.
Hughes, J. C., Graham, N., Patterson, K., & Hodges, J.
R. (1997). Dysgraphia in mild dementia of the
Alzheimer’s type. Neuropsychologia, 35, 533–545.
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
Kessler, B., & Treiman, R. (2001). Relationships
between sounds and letters in English monosyllables.
Journal of Memory and Language, 44, 592–617.
Kohonen, T. (1984). Self-organisation and associative
memory. New York: Springer.
Kreiner, D. S. (1992). Reaction time measures of spell-
ing: Testing a two-strategy model of skilled spelling.
Journal of Experimental Psychology: Learning, Memory
and Cognition, 18, 765–776.
Kreiner, D. S. (1996). Effects of word familiarity and
phoneme–grapheme polygraphy on oral spelling
time and accuracy. The Psychological Record, 46, 49–
70.
Kreiner, D. S., & Gough, P. B. (1990). Two ideas about
spelling: Rules and word-specific memory. Journal of
Memory and Language, 29, 103–118.
Kucera, H., & Francis, W. N. (1967). Computational
analysis of present-day American English. Providence,
RI: Brown University Press.
Lambert, J., Eustache, F., Viader, F., Dary, M., Rioux,
P., & Lechevalier, B. (1996). Agraphia in Alzhei-
mer’s disease: An independent lexical impairment.
Brain and Language, 53, 222–233.
Lambon Ralph, M., Ellis, A. W., & Franklin, S. (1995).
Semantic loss without surface dyslexia. Neurocase, 1,
363–369.
Lashley, K. S. (1951/1960). The problem of serial order
in behaviour. In F. Beach, D. Hebb, C. Morgan, &
H. Nissen (Eds.), The neuropsychology of Lashley (pp.
506–528). New York: McGraw-Hill.
Link, K., & Caramazza, A. (1994). Orthographic struc-
ture and the spelling process: A comparison of differ-
ent codes. In G. D. A. Brown & N. C. Ellis (Eds.),
Handbook of spelling: Theory, process and intervention
(pp. 261–294). Chichester, UK: John Wiley.
Macoir, J., & Bernier, J. (2002). Is surface dysgraphia tied
to semantic impairment? Evidence from a case of
semantic dementia. Brain and Cognition, 48, 452–
247.
McCarthy, R., & Warrington, E. K. (1986). Phonologi-
cal reading: Phenomena and paradoxes. Cortex, 22,
359–380.
McClelland, J. D., & Elman, J. L. (1986). The TRACE
model of speech perception. Cognitive Psychology, 18,
1–86.
McClelland, J. L., & Rumelhart, D. E. (1981). An inter-
active activation model of context effects in letter
perception: Part 1. An account of basic findings.
Psychological Review, 88, 375–407.
157
 HOUGHTON AND ZORZI
Miller, R. R., Barnet, R. C., & Grahame, N. J. (1995)
Assessment of the Rescorla-Wagner model. Psycho-
logical Bulletin, 117, 363–386.
Morais, J., Cary, L., Alegria, J., & Bertelson, P. (1979).
Does awareness of speech as a sequence of phonemes
arise spontaneously? Cognition, 7, 323–331.
Morton, J. (1980). The logogen model and orthographic
structure. In U. Frith (Ed.), Cognitive processes in spell-
ing (pp. 117–113). London: Academic Press.
Noble, K., Glosser, G., & Grossman, M. (2000). Oral
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
reading in dementia. Brain and Language, 74, 48–69.
Norris, D. (1994). A quantitative, multiple levels model
of reading aloud. Journal of Experimental Psychology:
Human Perception and Performance, 20, 1212–1232.
Olson, A., & Caramazza, A. (1994). Representation and
connectionist models: The NETspell experience. In
G. D. A. Brown & N. C. Ellis (Eds.), Handbook of
spelling: Theory, process and intervention (pp. 337–
364). Chichester, UK: John Wiley and Sons.
Patterson, K., & Behrmann, M. (1997). Frequency and
consistency effects in a pure surface dyslexic patient.
Journal of Experimental Psychology: Human Perception
and Performance, 23, 1217–1231.
Patterson, K. E., & Hodges, J. R. (1992). Deterioration
of word meaning: Implications for reading.
Neuropsychologia, 12, 1025–1040.
Paulesu, E., McCrory, E., Fazio, F., Menoncello, L.,
Brunswick, N., Cappa, S. F., Cotelli, M., Cossu, G.,
Corte, F., Lorossu, M., Pesenti, S., Gallagher, A.,
Perani, D., Price, C., Frith, C. D., & Frith, U. (2000).
A cultural effect on brain function. Nature Neurosci-
ence, 3, 91–96.
Penniello, M. J., Lambert, J., Eustache, F., et al. (1995).
A PET study of the functional neuroanatomy of writ-
ing impairment in Alzheimer’s disease. The role of
the left supramarginal and left angular gyri. Brain,
118, 697–706.
Pinker, S. (1997). Words and rules in the human brain.
Nature, 387, 547–548.
Platel, H., Lambert, J., Eustache, F., Cadet, B., Dary,
M., Viader, F., & Lechevalier, B. (1993). Character-
istics and evolution of writing impairment in Alzhei-
mer’s disease. Neuropsychologia, 31, 1147–1158.
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., &
Patterson, K. E. (1996). Understanding normal and
impaired word reading: Computational principles in
quasi-regular domain. Psychological Review, 103, 56–
115.
Plaut, D. C., & Shallice, T. (1993). Deep dyslexia: A case
study of connectionist neuropsychology. Cognitive
Neuropsychology, 10, 377–500.
158 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
Plunkett, K., & Marchman, V. (1991). U-shaped learn-
ing and frequency-effects in a multilayered
perceptron—Implications for child language acquisi-
tion. Cognition, 38, 43–102.
Price, C. J., Moore, C. J., Humphreys, G. W., & Wise,
R. J. S. (1997). Segregating semantic from phonolog-
ical processing during reading. Journal of Cognitive
Neuroscience, 9, 727–733.
Rapcsak, S. Z., Arthur, S. A., Bliklen, D. A., & Rubens,
A. B. (1989). Lexical agraphia in Alzheimer’s disease.
Archives of Neurology, 46, 65–68.
Rapcsak, S. Z., Arthur, S. A., & Rubens, A. B. (1988).
Lexical agraphia from focal lesion of the left
precentral gyrus. Neurology, 38, 1119–1123.
Rapp, B., Epstein, C., & Tainturier, M. J. (2002). The
integration of information across lexical and
sublexical processes in spelling. Cognitive
Neuropsychology, 19, 1–29.
Rapp, B., & Goldrick, M. (2000). Discreteness and
interactivity in spoken word production. Psychological
Review, 107, 460–499.
Roeltgen, D. P., & Heilman, K. M. (1984). Lexical
agraphia: Further support for the two-system hypo-
thesis of linguistic agraphia. Brain, 107, 811–827.
Rosenblatt, F. (1962). Principles of neurodynamics. New
York: Spartan.
Rumelhart, D. E., Hinton, G. E., & Williams, R. J.
(1986). Learning internal representations by error
propagation. In D. E. Rumelhart & J. L. McClelland
(Eds.), Parallel distributed processing: Exploration in
the microstructure of cognition. Volume 1: Foundations
(pp. 318–362). Cambridge, MA: MIT Press.
Schwartz, M. F., Saffran, E. M., & Marin, O. S. M.
(1980). Fractionating the reading process in demen-
tia: Evidence for word-specific print-to-sound associ-
ations. In M. Coltheart, K. E. Patterson, & J. C.
Marshall (Eds.), Deep dyslexia (pp. 259–269).
London: Routledge & Kegan Paul.
Seidenberg, M. S., & McClelland, J. L. (1989). A
distributed, developmental model of word recogni-
tion and naming. Psychological Review, 96, 523–568.
Seidenberg, M. S., Waters, G. S., Barnes, M. A., &
Tanenhaus, M. K. (1984). When does irregular spell-
ing or pronunciation influence word recognition?
Journal of Verbal Learning and Verbal Behaviour, 23,
383–404.
Sejnowski, T. J., & Rosenberg, C. R. (1987). Parallel
networks that learn to pronounce English text.
Complex Systems, 1, 145–168.
Shallice, T. (1981). Phonological agraphia and the lexical
route in writing. Brain, 104, 413–429.

Shallice, T. (1988). From neuropsychology to mental struc-
ture. Cambridge: Cambridge University Press.
Shallice, T., Glasspool, D. W., & Houghton, G. (1995).
Can neuropsychological evidence inform connection-
ist modelling? Analyses of spelling. Language and
Cognitive Processes, 10, 195–225.
Shanks, D. R. (1991). Categorization by a connectionist
network. Journal of Experimental Psychology: Learning,
Memory and Cognition, 17, 433–443.
Stone, G. O., Vanhoy, M., & Van Orden, G. C. (1997).
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013
Perception is a two-way street: Feedforward and feed-
back phonology in visual word recognition. Journal of
Memory and Language, 36, 337–359.
Sutton, R. S., & Barto, A. G. (1981). Toward a modern
theory of adaptive networks: Expectation and
prediciton. Psychological Review, 88, 135–170.
Taraban, R., & McClelland, J. L. (1987). Conspiracy
effects in word pronunciation. Journal of Memory and
Language, 26, 608–631.
Thomas, M., & Karmiloff-Smith, A. (in press). Are
developmental disorders like cases of adult brain
damage? Implications from connectionist modelling.
Behavioural and Brain Sciences.
Treiman, R. (1986). The division between onsets and
rimes in English syllables. Journal of Memory and
Language, 25, 476–491.
Treiman, R. (1993). Beginning to spell: A study of first
grade children. New York: Oxford University Press.
Treiman, R. (1994). Sources of information used by
beginning spellers. In G. D. A. Brown & N. C. Ellis
(Eds.), Handbook of spelling: Theory, process and inter-
vention (pp. 75–92). Chichester, UK: John Wiley.
Treiman, R., & Barry, C. (2000). Dialect and
authography: Some differences between American
and British spellers. Journal of Experimental Psychol-
ogy: Learning, Memory and Cognition, 26, 1423–1430.
Treiman, R., & Chafetz, J. (1987). Are there onset- and
rime-like units in printed words? In M. Coltheart
(Ed.), Attention and performance XII: The psychology of
COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)
CONNECTIONIST DUAL-ROUTE SPELLING
reading. Hove, UK: Lawrence Erlbaum Associates
Ltd.
Treiman, R., Kessler, B., & Bick, S. (in press). Context
sensitivity in the spelling of English vowels. Journal of
Memory and Language.
Treiman, R., Mullenix, J., Bijeljac-Babic, R., & Rich-
mond-Welty, E. D. (1995). The special role of rimes
in the description, use, and acquisition of English
orthography. Journal of Experimental Psychology:
General, 124, 107–136.
Ward, J., Stott, R., & Parkin, A. J. (2000). The role of
semantics in reading and spelling: Evidence for the
“summation hypothesis”. Neuropsychologia, 38, 1643–
1653.
Weekes, B., & Coltheart, M. (1996). Surface dyslexia
and surface dysgraphia: Treatment studies and their
theoretical implications. Cognitive Neuropsychology,
13, 277–315.
Widrow, G., & Hoff, M. E. (1960). Adaptive switching
circuits. Institute of Radio Engineers, Western Electronic
Show and Convention Record, Part 4 (pp. 96–104).
Ziegler, J. C., Stone, G. O., & Jacobs, A. M. (1997).
What is the pronunciation for -ough and the spelling
for /u/? A database for computing feedforward and
feedback consistency in English. Behavior Research
Methods, Instruments, and Computers, 29, 600–618.
Zorzi, M. (2000). Serial processing in reading aloud: No
challenge for a parallel model. Journal of Experimental
Psychology: Human Perception and Performance, 26,
847–856.
Zorzi, M., Houghton, G., & Butterworth, B. (1998a).
Two routes or one in reading aloud? A connectionist
dual-process model. Journal of Experimental Psychol-
ogy: Human Perception and Performance, 24, 1131–
1161.
Zorzi, M., Houghton, G., & Butterworth, B. (1998b).
The development of spelling–sound relationships in a
model of phonological reading. Language and Cogni-
tive Processes, 13, 337–371.
159
 HOUGHTON AND ZORZI

APPENDIX A
a
Stimuli used in Simulations 3 and 7

Regular consistent words Exception words

——————————— ———————————
High Low High Low

best deft are bowl

big broke both broad
came bap break bush
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

clasp dig choose deaf

dark dot come doll
did bake do flood
got float dough gross
him grape done lose
page lunch foot pear
place pill front phase
soon pitch give pint
still pump great plough
stop ripe have rouse
tell sank move sew
will slam pull shoe
with slip put spook
think stunt said swarm
bring swore shall swamp
cost trunk want touch
felt wake watch wad
large wax were wand
land weld what wash
north wing word wool
press wit work worm
a
The word lists are based on those used in the Taraban and
McClelland (1987) study of reading, but the regular
consistent words have been adapted somewhat to conform to
sound-to-spelling consistency.

160 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)

 CONNECTIONIST DUAL-ROUTE SPELLING

APPENDIX B
a
Lists of irregularly spelled words used in Simulations 8, 9, and 10

Word frequency band

—————————————————————————————————————
0-9 10-19 20-49 50-99 100-199 200+

womb tomb debt wild doubt sure

chrome ache scheme laugh blood give
chef glove tour spread stood move
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

yacht gauge grow height gone come

awe deaf scarce broad heart are
dove wool breath foot month child
shone brow grind key meant kind
hind plough blind cook blue said
dread soup aunt shook talk death
sweat sword guide lose due head
yearn dough breast bought floor friend
sieve ton thread learn dead book
soot shoe wealth build wrote good
crook pint realm knife truth how
hook tough sign road now
brook rough style live could
vow touch should
sponge myth group
suave guard young
cough bread build
trough two
shove both
front

5 12 35 70 156 825
a
Stimuli from word sets used by Behrmann and Bub (1992, Appendix 3). Each column contains
words from one frequency band, and the mean frequency of each set is given at the bottom of
the column.

COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2) 161

 HOUGHTON AND ZORZI

APPENDIX C

a
Lists of regularly spelled words used in Simulations 8, 9, and 10

Word frequency band

——————————————————————————————————————–
0-9 10-19 20-49 50-99 100-199 200+

chore curb branch chest charge point

preach hub grave frame leave five
Downloaded by [b-on: Biblioteca do conhecimento online UTAD] at 09:56 11 February 2013

chess pinch globe bond spoke those

chive arch shame list lost more
hant couch slide pitch wish save
punch slice cure goal fear think
sage hint carve proof peace firm
shave grove storm boot pool grove
skate dose milk round soon wait
sane ranch lost air ground reach
hive fist brush wage sound each
dole launch dean nine stand food
mole heap lean strike brown tool
mint teach teach moon shot show
wilt boot mount block hear house
spear glow bound hell green found
breach mouse trap ground
peach switch strip street
niece ditch yard black
broom song time
broon sweet just
coil clean made
rouse part
bounce state
gaze
5 17 44 73 189 530
a
See footnote to Appendix B.

162 COGNITIVE NEUROPSYCHOLOGY, 2003, 20 (2)