Muhammad Zaki Zakariyya

2000523
Article Summary
Title: Social Semiotic Multimodality
Authors: Carey Jewitt/Berit Henriksen
Section Main Points
A History of Social  Social semiotics is the study of meaning-making and the
Semiotics: From individuals and processes involved in meaning creation. It
Language to investigates the many media and strategies of communication
Multimodal Sign- that people employ to represent their worldview and shape power
making relations with others. It is based on a qualitative, fine-grained
examination of artifacts and texts as records of meaning-making
in various social and cultural settings.
 The passage traces the development of social semiotics as a
theoretical approach to multimodality. It starts with its origins in
Functional Linguistics, particularly in the work of Michael
Halliday.In Halliday's theory, language is shaped by social
processes to fulfill communicative needs.
 Robert Hodge and Gunther Kress extended Halliday's approach
from language to the broader concept of sign-making. They
argued that societies develop and shape all semiotic resources
(e.g., writing, images) to serve social functions, express values,
knowledge systems (discourses), and power structures.
 Kress and Theo van Leeuwen took the concept of meaning as
choice and applied it to visual communication. They
hypothesized the visual as a mode and investigated the grammar
of visual design in their book "Reading Images: The Grammar of
Visual Design." This broadening resulted in the concept of
multimodality, which emphasizes the relevance of numerous
modes in communication while also investigating shared
principles underlying multimodal communication.
 The passage mentions key works in multimodality, including
"Multimodal Discourse: The Modes and Media of Contemporary
Communication" by Kress and van Leeuwen (2001). It highlights
the exploration of multimodality from various perspectives and
the investigation of modes and multimodal representation.
 The passage concludes by noting that Kress's book
"Multimodality" (2010) and "Introducing Multimodality"
(Jewitt/Bezemer/O'Halloran 2016) consolidate the various
aspects of social semiotics and provide an approach to
understanding contemporary communication from a social
semiotic perspective.

Social Semiotics  Social semiotics provides a framework for understanding how

representations are produced and contribute to cultural contexts,
 focusing on their social function and meaning potential within
the communicative landscape.
 Social semiotics involves analyzing the textual features of
representations to comment on various aspects, including social
relations, power dynamics, signification, the interests of sign
makers, the imagined audience, and the social purposes achieved
by texts.
 Key Concepts in Social Semiotics:
a. Semiotic Resource: Semiotic resources are actions, materials, and
artifacts used for communicative purposes. These resources have
meaning potential based on their past uses and affordances based on
their possible uses in specific social contexts. Semiotic resources are
chosen by individuals in a socially regulated manner to express
meanings, emphasizing the concept of meaning as choice from a system.
b. Provenance: Provenance refers to the meaning potential of semiotic
resources derived from their previous uses in society. Semiotic resources
are constantly imported from other contexts, and this principle of
provenance plays a role in shaping the meaning of signs.
c. Modal Affordance: Modal affordance relates to what can be easily
expressed and represented with a mode. Affordances are shaped by the
material and cultural, as well as social historical use of a mode. Each
mode possesses a specific "logic" based on its materiality and cultural
conventions, affecting its communicational and representational
potentials.
d. The Motivated Sign and the Sign-Maker’s Interest: Signs are
considered motivated, meaning they are not arbitrarily associated but are
shaped by the sign-maker's interests. The choices made by sign-makers
are constrained by available modes and shaped by social conventions
and contexts.
e. Design: Social Semiotics examines the relationships between modes
in multimodal texts, recognizing that meaning is distributed across
modes in different ways. The interplay between modes and how they
contribute to the overall meaning of a multimodal text or interaction is a
central focus.
1. Signs, modes, and meaning-making are viewed as fluid,
dynamic, and open systems closely linked to their social context
of use. The analysis of modal systems, their resources, and
principles is rooted in the social and cultural context.
2. The passage indicates that these five concepts can be applied to
explore the multimodal construction of gender using Pinterest as
an example.

Applying a Social  The study utilizes a social semiotic multimodal approach to

Semiotic examine how gendered styles are constructed on the social media
Multimodal platform Pinterest. It aims to understand how Pinterest users
Approach: Style on make choices regarding various semiotic resources to express
Pinterest and materialize their style.
 Pinterest is introduced as a social media platform that allows
users to curate and share visual content and hyperlinks through
the organization of images on virtual pinboards. The platform
relies on user participation, such as pinning images to share them
with others.
 The study adopts a social semiotic multimodal analysis, which
involves examining three inter-connected levels: modes and
semiotic resources, design (inter-modal relations), and the sign-
maker and context. These levels are treated as analytically
discreet, forming a sequenced approach.
 : The research question revolves around the construction of
gendered style on Pinterest and explores the extent to which
social media platforms and user practices create or reinforce
gender-based norms.
 The analysis involves examining the homepages of two typical
Pinterest users, referred to as User 1 and User 2, whose profiles
match the demographic of typical Pinterest users (female, mid-
twenties). These homepages are used to investigate how
gendered style choices are materialized on Pinterest.
 The study describes the three stages of a social semiotic
multimodal analysis: analysis of modes and semiotic resources,
analysis of modal design, and analysis of the motivated sign and
sign maker's interests. It is explained that these stages help in
understanding how users construct style and the design principles
they apply.
 Pinterest users select images through processes like repinning,
adding pins, or uploading pins. The study traces the provenance
of images to understand how users build their collections and the
sites they browse. It's noted that repinning is more common than
uploading original images, emphasizing the importance of image
sources within Pinterest.
 Pinterest primarily features conceptual images that focus on
objects or settings, with people often being secondary or absent
from the images. The analysis highlights how users' image
choices and themes relate to common Pinterest themes, which
are largely gendered around topics like home-making, fashion,
and marriage.
 The study identifies color as a significant resource in Pinterest.
The choice of color in images, rather than their content, plays a
key role in image selection. Users often choose colors associated
with femininity, such as various shades of pink and pastels.
 Writing and typography are constrained but significant modes on
Pinterest. Users have limited space for text, and writing is used
to present personal interests and styles in the context of Pinterest.
Writing reflects certain conventions, including passive
 constructions and the absence of personal pronouns, enhancing
the focus on the images themselves.
 The study highlights that Pinterest's design imposes significant
constraints on users, limiting their control over modes such as
image size, font, and color. Pinterest aims to maintain a high
level of coherence and sameness among its users. Users seem to
accept this constraint in favor of the platform's coherence.
 Pinterest users construct highly gendered styles through their
choices of images, color palettes, and writing. The platform's
design and normative practices play a substantial role in
maintaining this gendered style. This is seen as significant in the
context of debates about how social media platforms either
enable or constrain users in expressing their identities.

The Potentials and 1. Potential of Social Semiotic Multimodal Approach:

Limitations of a  Social semiotic multimodal analysis is a powerful tool for
Social Semiotic understanding the social function and complexity of
Approach various texts.
 It emphasizes the interconnectedness between people's
agency, the technologies they use, and the social context
of meaning-making, enabling a holistic and situated
analysis of communicative and representational texts.
2. Understanding Texts as Social Action:
 Social semiotics views texts as a material trace of the
maker's social action and as complex signs that act in and
on the world.
 It focuses on the sign maker, their interests, choices, and
multimodal design, allowing for the theorization of the
connection between texts and their creators.
3. Examination of How Actors Use Modes:
 This approach enables the examination of how different
actors take up modes within specific environments and
explores the consequences of these actions for
communication, learning, and identity.
 It is sensitive to the exploration of power relations and
how they are materially instantiated in various contexts.
4. Contributions to Critique, Design, and Speculation:
 A social semiotic approach to multimodality contributes
not only to the critique of existing resources and practices
but also to the design and speculation about future
resources and their potential uses.
5. Limitations of the Approach:
 The approach focuses on a limited collection of texts,
raising concerns related to issues like selection, sampling,
consistency, and generalization.
 The emphasis on the sign maker raises questions about
 how intentionality can be evaluated and assigned, as well
as the interpretation of text.
 Critics have pointed out potential limitations, including a
focus on semiotic texts rather than practices, the creation
of binary oppositions, and strong theoretical
commitments.
 The approach's analytical reach beyond the text is
limited, and other social theories may be needed to
interpret the social implications of the text's content.
6. Overcoming Limitations through Complementary Methods:
 Combining a social semiotic multimodal approach with
other methods (e.g., focus groups, content analysis) that
deal with larger scales of data can help overcome some of
the limitations and bridge the gap between micro-analysis
and macro-social concerns.

Conclusion In conclusion, this chapter emphasized the power of social semiotic

multimodal analysis for understanding how users construct style on
social media platforms like Pinterest. It highlighted the role of social
norms and design affordances in shaping user practices and challenged
some common assumptions about social media as sites for image
creation and self-expression.