Semiotic Repertoires: Transforming

the Study of Languages

Reza Jafary mohamadreza.jafary@mail.mcgill.ca

Jinyi Liu jinyi.liu2@mail.mcgill.ca

Nishat Sharmin nishat.sharmin@mail.mcgill.ca

Main areas to be covered in presentation

● Sign Languages
● Linguistic Diversity & Multilingualism
● Multimodality Studies
Symbolism and Semiotics

● https://www.youtube.com/watch?v=88129wxFLuo
Auld Lang Syne
"I have met a lot of friends online, which is great. One
guy randomly stumbled upon my profile page and we
really had a lot of stuff in common. Both had really
similar interests and the conversation was so fluid and I
could feel a friendship building. He was a bit
apprehensive to tell me after talking a lot for a week or
so, but he finally came around to telling me that he is
deaf! He was worried I would be off-put but when he
heard that I took ASL as my foreign language, he was so
excited to meet! We have hung out a few times and it
really great to still use ASL, especially with someone
who I can learn from every time we hang out. He is a
really great guy, deaf or hearing, and it feels great to still
practice everything I learned at OSU outside of the
classroom. ~Justin (Spring 2014, after completing the
series, graduating, and moving to NYC)
Gesture study/ repertoire
First, gestures can be seen as part of what
learners can acquire with a new language (“the
SLA of gestures”).

Second, learners’ gestures are interesting in and

of themselves for acquisition.

Finally, gestures can be seen as input, raising a

range of questions regarding their role in the
classroom, and their effects on comprehension
and learning more generally.

(Gullberg, 2006)
Sign languages

● Signs-the words of a sign language –are produced by actions of the hands,

arms, torso, face, and head that produce signals perceived visually.

● There are more than 300 different sign languages in use around the
world ， which vary from nation to nation.

(Liddell, 2003)
What is the difference between
signs and gestures?
Gesture

Add meaning

Pointing

Beats
Sign languages

Alternate sign languages is a system of hand signals developed by speakers for limited
communication in a specific context where speech cannot be used.

Primary sign languages is the first language of a group of people who do not use a
spoken language with each other.

（ Prasetyo, 2017)
 English
word
French sign

Multilingual
and
Multimodal
The reason why the sign languages
are endangered?
Coupling state-of-the-art hearing technologies with an exclusive
focus on spoken language acquisition.

The attrition of (sign bilingual)schools for the deaf as spaces for

the emergence and transmission of sign languages.

(Kusters, De Meulder, Friedner, & Emery, 2015)

My outline

● Monolingualism vs Multilingualism & Benefits of being a multilingual

● Moving from bilingualism to monolingualism
● The theme of multilingualism
● The main function of semiotic repertoire + it’s unique concept
● Multimodality of human interactions and linguistics repertoire
● Translanguaging, and the notion of semiotic assemblages
 ‫فارسی بلدی حرف‬
Sen Türkçe
Parlez-vous ‫بزنی؟‬
konuşmayı
français?
biliyor musun?

você fala
Parlez- vous
português?
espagnol?
Multilingualism as a Concept

“ Sim” , “Oui”,
If you “M‫”آره‬, “Evet”,
answered : “Jawohl”, “ or
“Si”
Monolingualism vs Multilingualism

Besides having an easier time traveling or watching

movies without subtitles, knowing two or more
languages means that your brain may actually look
and work differently than those of your monolingual
friends.
Benefits of being a
multilingual

Being a multilingual gave the brain

some remarkable advantages.

Some visible and some not visible

merits like:

higher density of gray matter

More activity in certain regions when

engaging in a second language
Recent studies were more promising

Recent studies showed that the amount of

effort and attention needed to switch between
languages triggered more activity in.

So, while bilingualism in case of Reza

[multilingualism, because her grandmother
used Azeri language on most occasions] did
not necessarily make him smarter, it did help
his brain to be healthy, complex and actively
engaged as my mummy says so .
 Transforming
Transforming the
the study
study of
of semiotic
semiotic repertoires
repertoires

Moving
Moving from
from bilingualism
bilingualism to
to multilingualism
multilingualism & & (trans)languaging
(trans)languaging
Revitalization
Revitalization of
of language
language repertoire
repertoire
Controversy:
Controversy: Most
Most multimodality
multimodality studies
studies the
the main
main focus
focus was
was on
on using
using one
one spoken
spoken
language
language within
within embodied
embodied human
human action.
action.
Communication
Communication was was multimodal
multimodal but
but not
not multilingual
multilingual
Opposing
Opposing concept
concept was
was dominant
dominant inin translanguaging:
translanguaging:
Communication
Communication was was multilingual
multilingual without
without paying
paying attention
attention to
to multimodality
multimodality ,,
simultaneity
simultaneity &
& hierarchies
hierarchies within
within the
the simultaneous
simultaneous combination
combination ofof resources.
resources.
Recent movement/ current status
of the study

The sociolinguistics of sign language has paid attention

to multimodality but only very recently have started to
fucus on multilingual contexts where multiple sign /or
multiple languages are used.
Little transaction between these areas of research.

The lens of semiotic repertoires enables synergies to be

identified and provides a holistic focus on action that is
both multilingual and multimodal.
The theme of multilingualism/ an over arching
context

Multilingualism comprises perspective from education and urban spaces. It embraces

the concept of repertoire as the totality of linguistic resources of the individual (Busch,
2012,2015; Spotti & Blommaert,2017).

Multilingualism considers the main concept of translanguaging as the individual’s

dynamic use of their linguistic resources in various contexts for meaning-making
irrespective of socio- cultural boundaries of names languages (Otheguy, Garcia, &
Reid, 2015).

Translanguaging transforms repertoires as resources are added, expanded, revised

and sometimes sedimented onto particular functions and within particular contexts
leading to ‘ spatial repertoires”, Pennycook Otsuji,2015).
The main concept of
repertoire in this study

● It traverses studies of spoken and signed language and

softens the boundaries between languages & research
paradigms.
● It gives a fresh perspective on the multimodal and
multilingual aspect of communication and a more
nuanced understanding of translanguaging by
recognizing various ways in which individuals draw on
their multimodal linguistic resources to make meaning.
Gesture studies & multimodality focused on monolingual utterances
rather than on multilingualism & linguistic diversity

Sign language in conjunction with

spoken language /in the context
of sign bilingualism/bimodal
bilingualism
These lines of enquiry are less
visible in multilingual context
where multiple sign or multiple
spoken language are used.
Bridging the gaps for the lack of
transactions
● The Lack of transactions does exist between research that focuses on
gestures, signs and multimodality and research into linguistic diversity or
multilingualism.
● Although some scholars like Garcia and Wei (2014) have emphasized
that translanguaging in essentially multimodal, they have not yest
expanded on this concept.
● Linguistic landscaping is inherently multimodal mostly focusing on
pictures, smells, signage, blackboards and screens and not so much on
visual-gestural modality of communication.

● Question: How to bridge all these various fields?

● Do you support the concept looking at language as a phenomenon

derived from linguistic repertoires as internalized individual
competence? Please justify your reason
“Semiotic repertoires a potential bridging
multilingualism and multimodality research.”

The concept of semiotic repertoire offers a way articulating the way

individuals draw on their diverse resources mapping them onto
functions in a communicative act.

It could encompass, music, signs, various genres even a smell and

any media for the expression of ideas.
The main function of semiotic repertoire
Translanguaging scholars focused on multi modality particularly via internet memes, social media or mobile phone
texting.

Linguistic landscaping ( Pennycook, 2017), as another branch of the study of language in society, is inherently multimodal,
but mostly by focusing on pictures, smells, signage, blackboards and screens; and not so much on the use of visual-
gestural modality of communication including signing, gesturing, body orientation and the use of objects.

Therefore, to bridge these various fields, we use semiotic repertoire.

Multimodality of human interactions and
linguistics repertoire

Multimodality scholars have

Language in use, spoken, signed
investigated how different ‘ modes’
or text, is always and inevitably
work together but there is still a
constructed across multiple
lack of transaction between
modes of communication and
research that focuses on gesture,
through ‘contextual’ phenomenon
signs and multimodality on the
such as the sue of surrounding
one hand, and research into
physical spaces ( Goodwin,2000;
linguistic diversity or
Scollon & Levine,2004).
multilingualism on the other hand.
 Repertoire: conceptual overview and re-
evaluation

● The concept of repertoire offers a way articulating the way individuals draw
on their diverse resources mapping them onto functions in a communicative
act.
● John Gumpez and Dell Hymes, in their “Directions in Sociolinguistics
( 1972,pp. 20-21)”, listed repertoire as one of the basic sociolinguistic
concepts and defined it as the totality of linguistic resources available to
members of particular communities.
 - Traverses studies of spoken and signed
The unique language and softens the boundaries between
languages and research paradigms.
concept of
repertoire
- Enables a fresh perspective on the multi modal
and multilingual aspects of communication.

- Redefines trasnlanguaging in a sense that

recognizes the different ways in which
individuals draw on their multimodal linguistic
resources to make meaning.
Translanguaging

Mixing languages

The crux of pedagogical trasnlanguaging : languages can work

together in the classroom rather than in competition

Translanguaging can be integrated into the classroom setting

through:

A. Varying the language input processing

B. Varying the language of output

Translanguaging and semiotic assemblages

Developing
Borders the idea of
between spatial
semiotic repertoires
modes and
assemblages

The need to
grasp the
A wide range
relationship
of semiotic
among a
possibilities
range of
semiosis
Can you name the modes of
communication?
Source:
https://inclusiveclassrooms.org/resources/practices/multimodality
What types of modes do you use in your everyday communication? Do
they convey similar meaning in different contexts? Why/Why not?

● https://jamboard.google.com/d/1eqxFeHyMCZePX6Y7EZWAPGk2SF
RgG77Kgs3BXnxLQQM/viewer?f=4
Multimodality Studies

Environmen
Resources Objects Actions
t
Resources

● Resources are mobilised ‘in response to the contingencies of the setting and of the interaction’ (Mondada, 2014, p.
140).
Objects

● While objects are everywhere around us when interacting, they are ‘made relevant through participants’ pointing,
referencing, naming and touching’ (Nevile et al., 2014, p. 15)
Actions

● Actions consist of different consecutive steps, and simultaneous presence of particular semiotic fields (such as: a
particular body orientation, a point or an emblematic gesture and a spoken utterance) could last just a few seconds,
until the next ‘stage’ of the action.
Environment

● Role of surroundings
● Restructured and reorganized while intersecting with actions, resources, and simultaneous presence of semiotic
fields
Modal Intensity & Modal
Complexity

● Modes analyzed in terms of intensity and complexity

Conclusion

●
Concept of “semiotic repertoires” is in line with the notion of distributed language, cognition and agency which allow

us to see how they are produced in material webs of human and non-human assemblages. Looking at languages in

these terms help us see that meaning- as radically indeterminate signs emerges from interaction rather than considering

linguistic repertoires as internalized individual competence. Semiotic repertoires bridges all types of modalities in one

integrated holistic system.

Take Home Message

● Allows for an understanding of how different trajectories or people, semiotic resources and objects meet at
moments and places alongside the meanings of linguistic resources (e.g., shop assemblages)
References

Liddell, S. K. (2003). Grammar, gesture, and meaning in American Sign Language.

Cambridge University Press.

Prasetyo, A.(2017, January 10). Gestures And Sign Languages. Linguistik.

https://www.linguistikid.com/2017/01/gestures-and-sign-languages.html

https://www.youtube.com/watch?v=fpZ3qxTW3cE

Pictures for Multimodality

https://inclusiveclassrooms.org/resources/practices/multimodality
References

● Kusters, A., Spotti, M., Swanwick, R., & Tapio, E. (2017). Beyond languages, beyond
modalities: Transforming the study of semiotic repertoires. International Journal of
Multilingualism, 14(3), 219-232.
● Pennycook, A. (2017). Translanguaging and semiotic assemblages. International
Journal of Multilingualism, 14(3), 269-282.
● Otheguy, R., García, O., & Reid, W. (2015). Clarifying translanguaging and
deconstructing named languages: A perspective from linguistics. Applied
Linguistics Review, 6(3), 281-307.
● Pennycook, A., & Otsuji, E. (2015). Metrolingualism: Language in the city. Routledg
● Moore, E., & Bradley, J. (2020). Resemiotisation from page to stage: translanguaging
and the trajectory of a musilingual youth’s poem. International Journal of Bilingual
Education and Bilingualism, 23(1), 49-64.
● Blom, J.-P., Gumperz, J., & Hymes, D. (1972). Directions in sociolinguistics.