Understanding and evaluating multimodal texts  Visuals are better at giving a sense of size, color,

space, etc. When looking at a picture, you do not

MODES have to start from left to right or top to bottom.
Examples of modes are: You are free to explore the space as you “read”
 Reading the image.
 Writing
 Listening What is even better is that using two or more of these
 Speaking modes of representation together can enrich our
 viewing understanding of a topic as they can be designed to
act as complementary to each other.
When we combine…..
 Picture book
 VISUAL IMAGES What is audio description?
 PRINT TEXT Audio description involves the accessibility of the
 SPEECH IN A FILM visual images of theater, television, movies, and other
art forms for people who are blind, have low vision,
Multimodal TEXT or who are otherwise visually impaired.
 A text may be defined as multimodal when it
combines two or more semiotic systems. There High-context vs. low context continuum
are five semiotic systems in total. Concept used in communication studies to describe
cultures based on the amount of contextual
The five multimodal text information needed for understanding
1. Linguistic-Comprising aspects such as
vocabulary, generic structure and the grammar of HIGH CONTEXT
oral and written language.  Rely heavily on implicit communication, shared
2. Visual-Comprising aspects such as color, vectors understanding, and non-verbal cues.
and viewpoint in still and moving images. Example
3. Audio-Comprising aspects such as volume, pitch Japan, China, and many middle eastern countries. For
and rhythm of music and sound effects. japan, silence can convey a lot of meaning, and
4. Gestural-Comprising aspects such as movement, indirect communication is common.
speed and stillness in facial expression and body
language. LOW CONTEXT
5. Spatial-Comprising aspects such as proximity,  Rely more on explicit communication, with the
direction, position of layout and organization of messages conveyed primarily through words
objects in space. rather than context.
Example
EXAMPLES OF MULTIMODAL TEXTS In US, Germany, and many Western European
 A PICTURE BOOK countries, in these cultures, directness and clarity in
 A WEBPAGE communication are valued.
 A LIVE BALLET PERFORMANCE
Multimedia text can be delivered via different media
or technologies. They may be live, paper, or digital
electronic.

Each mode of the five semiotic systems has its own

rules and constraints.
For example:

 Alphabetic texts are great at sending a message

across, but they can also lead to multiple
interpretations or ambiguity at times.