Psycho- and cognitive Linguistics

Sign language - how does it differ from natural language?

1. Development of sign language linguistics

History

 Brief timeline:
360 BC: first mention of sign language
1620: first book about sign language
1750: first school for deaf children
1880: first association of deaf people in the US
1960: first modern monographic study of sign language
1995: Uganda as the first country to recognize sign language constitutionally
 Oral movement: oral methods of teaching for deaf children with no sign language (Alexander
Graham Bell, 1867) -> forced deaf children to communicate via spoken language

2. Structure of ASL
 Shared features with spoken language
o Language composed of symbols for an entity in the real world that its users
manipulate to produce meaning
o Organized symbols
o Arbitrariness and iconicity of symbols
o Showing relations between meanings
o Language can introduce new symbols

3. Differences in cognitive processes

Production

 Signs take longer to produce than words

 In order to accommodate to the lower word-rate, sign language makes use of several
aspects to help with faster communication
o Exclusion of …

Function words “It’s against the law to drive on the left side”
Anaphoras (anaphoric “John likes bread so John buys bread” ->
pronouns) “John likes bread so he buys bread”
General and specific verbs General verb: to have a bath
Specific verb: to bathe

o Incorporation of…

Location referred to by changing the

direction/movement of the sign
Number agreement in number between subject and
verb do not have to be specified, number is
only incorporated in the verb
Manner different connotations of signs can be shown
through the manner of signing
Size and shape Sign varies depending on the physical
appearance of the referents in the real world
Psycho- and cognitive Linguistics

o Body movements and facial expression

Facial expressions Negative expressions:

headshake/frowning
Body attitude Use of space for narrative
functions

The syllables of sign language

 Sign language and spoken language work with similar cognitive mechanisms and
neuroanatomical substrates
 Syllables in spoken language are the functional units during the planning of speech
o Locus: during encoding level or articulatory preparation
o Minimal phonological constituents of spoken language
o They include a nucleus and may include an onset or coda
 Sign language seems to have similar units
o Minimal phonological constituents of sign language
 Location
 Movement
 Handshape
o Signs are organized in static and dynamic alterations (corresponding to consonants
and vowels)
 Static alterations correspond to consonants
 Dynamic alterations correspond to vowels
 Movement has the status of the nucleus
 Location-Movement composes a syllable

Perception

 Iconicity
iconic and arbitrary signs are processed similarly by the brain
 Sequence of identification (Study by Emmorey and Corina, 1990)
sub-lexical elements appear (rather) simultaneously (in contrast to spoken/written words)
→ Location of a sign was identified first, followed by handshape and finally the movement
 Word onsets vs sign onsets (Study by Emmorey and Corina, 1990)
→ signs were identified more quickly
o spoken English words: ca. 83 % of a word had to be heard before identification
occurred
o signs: ca. 35 % of the sign form had been seen before identification occurred
o explanation:
 relatively greater simultaneous packaging of phonological information within
a sign
 few signs share an initial phonological shape, leading to a reduced number of
competing lexical items (i.e., reduced initial cohort size)
Psycho- and cognitive Linguistics

→ word onsets may be less informative than sign onsets

 Factors affecting sign recognition

Lexicality effects (Corina & Emmorey, 1993)

decision tasks: reaction times were significantly slower to formationally possible but non-
existing ASL signs than to real ASL signs

Co-articulatory factors (Emmorey & Corina, 1990)

signers could anticipate a handshape change prior to the full articulation of the handshape
→ may assist word identification

Lexical access (Caselli and Cohen-Goldberg, 2014)

o studied whether the mental lexicon is organized similarly across modalities and
whether words and signs are activated and selected in similar ways
o neighbours in sign language
sharing 1 sub-lexical unit
 “location neighbours”
 “handshape neighbours”
o Neighbourhood effects
 location neighbors inhibit lexical access
 handshape neighbors facilitate access
→ Factors affecting spoken word recognition also influence the recognition of signs.
→ Studies suggest that the mind stores and accesses words in the same manner,
no matter the modality.

The brain
 Brain areas
classical view:
o Left hemisphere (incl. Broca’s area) specialized for linguistic processing
o Right hemisphere specialized for visual-spatial abilities

→ What regions are activated when sign languages are used?

Meta-analysis from Max Planck Institute for Human Cognitive and Brain Sciences:
o Broca's area (left hemisphere) is also the crucial brain region for sign languages
o Right frontal brain (right hemisphere)

 processes non-linguistic aspects such as spatial or social information

 movements of the hands, face and body are in principle perceived similarly
by deaf and hearing people

 but deaf people additionally activate the language network in the left
hemisphere (including Broca's area)
Psycho- and cognitive Linguistics

→ they perceive gestures as something with linguistic content instead of pure

movement sequences

→ Broca's area is central for language no matter the modality

 Cross-modal plasticity
= reorganization of the brain's neurons to adapt to long-term sensory deprivation
Finney et al. (2001):
o early deafness results in the processing of visual stimuli in auditory cortex
o studies suggest that removal of one sensory modality leads to neural reorganization
of the remaining modalities
→ “unused” auditory cortex can be colonized by vision
Impact of SL on cognitive development: Theories of Mind

Theories of Mind:
 capability to understand that others have beliefs, desires, intentions, and perspectives that
are different from one's own
 coordination of visual perspective is one of the basic skills a child learns concerning ToM
 tested for example with “three mountain test” (Jean Piaget) or false-belief tasks (e.g. Sally-
Anne-task)

Study using false-belief tasks (Courtin, 2000):

 compared hearing children, deaf children of deaf parents, signing deaf children of hearing
parents and oral deaf children of hearing parents
 result: the deaf children of deaf parents outperformed all the other children
 possible explanation: role of perspective-taking in sign language fosters coordination of
visual perspectives necessary for theories of mind

Summarized comparison:

spoken/written languages sign languages

iconicity iconicity is not an advantage

identification more linear identification sequence: location, handshape, movement

~ 83 % heard before identification

onsets → word onsets may be less ~ 35 % seen before identification
informative than sign onsets

recognition factors (lexicality, neighborhood density, etc) influence recognition in both languages

mental lexicon the mind stores and accesses words in the same manner, no matter the modality

brain structure (currently) no differences found but: cross-modal plasticity

but: LH also activated when visual stimuli (gestures) are

Broca's area (LH) is central for perceived
brain activation
language no matter the modality
greater involvement of the brain's RH
Psycho- and cognitive Linguistics

cognitive some advantages regarding Theories of Mind:

development better/earlier understanding of visual perspectives

4. Bibliography

https://www.nidcd.nih.gov/health/american-sign-language

Baker-Shenk Charlotte, Cokely Dennis. American Sign Language. A Teacher’s Resource Text on Grammar and Culture.
Washington D:C.: Gallaudet University Press, 1991.

Bauman, Dirksen (2008). Open your eyes: Deaf studies talking. University of Minnesota Press. 

Baus, Christina, Gutiérrez, Eva, Carreiras, Manuel, The role of syllables in sign language production, in: Frontiers in
Psychology, 2014 https://www.frontiersin.org/articles/10.3389/fpsyg.2014.01254/full (10. Jun 2021)

Bavelier et al. “Children Need Natural Languages, Signed or Spoken.” Cerebrum. 2003.
https://dana.org/article/children-need-natural-languages-signed-or-spoken/#:~:text=Signed%20vs.,sign%20languages
%20and%20spoken%20languages.&text=Although%20the%20visual%20form%20of,%2C%20like%20words%2C%20are
%20conventionalized (15.06.2021)

Bellugi, Ursula u. Fischer, Susan, A comparison of sign language and spoken language, 1972
https://www.researchgate.net/publication/222508819_A_comparison_of_sign_language_and_spoken_language 

Campbell et al. “Sign Language and the Brain: A Review“. The Journal of Deaf Studies and Deaf Education. Vol. 13, Issue 1,
Winter 2008.
https://academic.oup.com/jdsde/article/13/1/3/500594 (15.06.2021)

Caselli, Naomi K. and Cohen-Goldberg, Ariel M. “Lexical access in sign language: a computational model”. 2014.
https://www.frontiersin.org/articles/10.3389/fpsyg.2014.00428/full (13.06.2021)

Corina, David P. and Knapp, Heather P. “Psycholinguistic and Neurolinguistic Perspectives on Sign Languages”. Handbook of
Psycholinguistics, 2006.

Courtin, Cyril. “Impact of Sign Language on the Cognitive Development of Deaf Children: The Case of Theories of Mind”.
Journal of Deaf Studies and Deaf Education. Vol 5, Issue 3, Summer 2000.
https://www.jstor.org/stable/42659285?seq=1#metadata_info_tab_contents (13.06.2021)

Finney et al. “Visual stimuli activate auditory cortex in the deaf”. Nature Neuroscience. 2001.
https://pubmed.ncbi.nlm.nih.gov/11704763/ (13.06.2021)

Griffith et al. “Perception of iconicity in American sign language by hearing and deaf subjects”. Journal of Speech and
Hearing Disorders. Vol. 46, Issue 4, Nov 1981. 
https://pubmed.ncbi.nlm.nih.gov/7300266/  (17.06.2021)

History magazine. How monks helped invent sign language. Inés Antón Dayas. Published may 28, 2019. IN The history of
sign language (nationalgeographic.com). Consulted: 30.05.2021

Max-Planck-Gesellschaft. “How the brain processes sign language”. 2021. https://www.mpg.de/16461250/0219-nepf-

language-is-more-than-speaking-how-the-brain-processes-sign-language-149575-x (14.06.2021)

Pfau Roland, Steinbach Marcus, Woll Bencie. Sign language : an international handbook. Berlin/Boston: De Gruyter Mouton,
2012.

Rutkowski Pawel. Signs and Structure. Formal Approaches to Sign Language Syntax.Amsterdam: John Benjamins Publishing
Company 2015.

Sun Kim, C. TED Talk “The enchanting music of sign language.”

https://www.youtube.com/watch?v=2Euof4PnjDk (17.06.2021)

Valli Clayton, Lucas Ceil. Linguistics of American SIgn Language. An Introduction. Washington DC: Gallaudet University Press,
2002.