1032965

research-article2021
NMS0010.1177/14614448211032965new media & societyLogi and Zappavigna

Article

new media & society

A social semiotic perspective

1­–25
© The Author(s) 2021
Article reuse guidelines:
on emoji: How emoji and sagepub.com/journals-permissions
DOI: 10.1177/14614448211032965
https://doi.org/10.1177/14614448211032965
language interact to make journals.sagepub.com/home/nms

meaning in digital messages

Lorenzo Logi and Michele Zappavigna

UNSW, Australia

Abstract
This article presents a social semiotic analysis of emoji-language semiosis. Combining
the theoretical architecture of Systemic Functional Linguistics and methodology of
Multimodal Discourse Analysis, we propose an analytical framework that can identify
how emoji make meaning both individually and in interaction with language. Using the
web-based coding software WebAnno, we apply this framework to a dataset of text
messages and social media posts. The results identify typical realisations of particular
semiotic features by emoji as well as noteworthy dynamics in how emoji interact
with language to realise meaning. We observe (1) how emoji and language jointly
construing ideational meaning realise intermodal taxonomies (where hyper/hyponyms
are distributed across modes) and particular fields of discourse (domains of experiential
meaning), (2) how resources in one mode can serve to foreground particular regions
of meaning potential in other modes, and (3) how attitudinal meaning realised by emoji
appears to differ from the prosodic patterning of linguistic attitude.

Keywords
Emoji, intermodality, Systemic Functional Linguistics, social semiotics

Introduction
There is currently extensive interest in linguistics and communication studies in the role
that non-linguistic features such as emoji play in digitally mediated communication.
Emoji are widespread in social media discourse, involved in the expression of emotion,

Corresponding author:
Michele Zappavigna, UNSW, Sydney, NSW, 2052, Australia.
Email: m.zappavigna@unsw.edu.au
2 new media & society 00(0)

conveying stances, and negotiating interpersonal alignments. This study draws on the
theoretical orientation of social semiotics to explore the kinds of meanings that can be
made as emoji interact with language (the linguistic ‘co-text’) in a social media post or
digital message.
Emoji are a subcategory of what Herring and Dainas (2017) classify as ‘graphicons’
(graphical icons) – a classification of graphical features found on social media platforms
that also include emoticons, stickers, graphics interchange formats (GIFs), images and
videos. Emoji are small images (e.g. ) represented using code points (a numerical code
assigned to each character) in the continually evolving Unicode Standard. They appear
differently depending on how they are renderd within different operating systems, plat-
forms, and software applications. Due to the operating system and software used to write
this paper, emoji are presented either as their Microsoft representations, which may be
different to the visual rendering in the original messages in the dataset. Common Locale
Data Repository (CLDR) shortnames are also used throughout (e.g. ‘Smiling Face with
Heart-Eyes’1), but should not be confused with analysis of the meaning of particular
emoji in context.
While the first recorded case of emoticon use occurred in 1982 on the Carnegie
Mellon bulletin board, emoji were created in Japan in 1997. Since then, emoji have been
widely adopted by computer-mediated-communication users: in 2016, almost 50% of
messages sent through messaging apps contained emoji (Emogi Research Team, 2016).
As emoji have become more popular, users have sought to expand their meaning poten-
tial and the Unicode Consortium has accepted many proposals to expand the emoji cata-
logue with more varied experiences, such as representation of a greater variety of skin
tones, genders, family types and regional foods. However, the skin tone modifiers have
been critiqued as enacting ‘an institutionalized form of colour-blind racism which insists
that concerns regarding racial representation and identity are irrelevant to “neutral” tech-
nical systems and workplaces’ (Miltner, 2021: 517). In 2021, the Unicode Standard con-
tained over 3300 emoji. Emoji thus constitute a vital area of linguistic investigation, both
due to their increasing usage and to their evolving meaning-making potential.
In seeking to advance our understanding of emoji, this article elaborates and applies
an analytical framework for exploring how emoji interact with language. Applying this
framework to the dataset suggests both typical meanings realised by specific emoji and
the high degree to which emoji meaning is context-sensitive. In regard to the latter point,
this study applies Zhao’s (2010, 2011) minimum mapping hypothesis, originally devel-
oped for understanding relations between images and language, to interpret how emoji
interact with language. The study proposes that the meaning jointly construed by emoji
and language is an area of shared meaning potential across communicative modes.
The analysis presented in this article is underpinned by the theoretical framework of
Systemic Functional Linguistics (SFL). Relevant principles of SFL will be outlined in
the third section, ‘Systemic Functional Linguistics (SFL)’, however, a brief glossary of
terms used in the article is provided here. These include the three metafunctions concep-
tualised in SFL theory: the ideational metafunction, concerned with experiential mean-
ing, the interpersonal function, concerned with enacting relationships, and the textual
function, describing resources for organising meaning into a coherent text (Halliday,
1978); and the systems that describe attitudinal meaning (attitude): affect (emotional
Logi and Zappavigna 3

states), appreciation (aesthetic evaluations) and judgement (social/moral evaluation).

Finally, ‘co-text’ refers to semiotic resources occurring within the same text.

Linguistic research on emoji

Semiotic research on emoji to date has evolved from focusing on their attitudinal meaning
to recognising their semantic, pragmatic and phatic function within texts (c.f. Danesi,
2016), often drawing parallels with how gesture contributes to semiosis (c.f. Gawne and
McCulloch, 2019). Outcomes of this work relevant to this study will be reviewed below.
An alternate avenue of enquiry has investigated the sociolinguistic features of emoji,
exploring how emoji use is distributed across text genres, and by age, gender, race and
cultural context (see Albawardi, 2018; Herring and Dainas, 2020; Miltner, 2021;
Nishimura, 2015). This is undoubtedly a rich and valid field of enquiry; however, its
results are not immediately relevant to this study, thus it will not be reviewed in any detail.

Pragmatic approaches
A primary issue to address in reviewing literature on pragmatic approaches to emoji
semiosis is the relationship between emoji and emoticons. Emoticons and emoji are dis-
tinct types of graphicons; however, the terms have somtimes been used interchangeably,
possibly as a reflection of a common software feature that automatically converts some
emoticons into emoji (e.g. in Microsoft Word). While the richer pictorial affordances of
Unicode afford emoji a greater meaning potential than is available to ASCII emoticons,
research which discusses emoticons is nevertheless relevant to this study. This reflects
the functional, intermodal approach taken here, which follows Albert (2020) in grouping
different categories of graphicons as semiotic resources interacting with language.
As surveyed in Dresner and Herring (2014), early work on emoticons (Rezabek and
Cochenour, 1998; Walther and D’addario, 2001) described their semiotic function as
constrained to affective meaning, a view summed up in Crystal’s (2001) description of
emoticons as ‘combinations of keyboard characters designed to show an emotional facial
expression’ ( p. 36). More recent work disputes this definition, arguing that, further to
attitudinal meaning, emoticons have phatic and pragmatic functions. Primary exponents
of this view are Dresner and Herring, who point to the multiple, context-dependent func-
tions emoticons can realise, and challenge the assumption that emoticons can make
meaning independent of co-text. In their work, Herring and Dainas (2017) identify six
main functions of graphicon use in Facebook comments, ‘reaction, tone modification,
riffing, action, and narrative sequence’ (p. 2185).
Many studies apply a pragmatic lens to emoji analysis in order to understand their
interpersonal functions. Sampietro (2016) argues that emoji are used by writers to ‘align
with the interlocutor, to express informality or to enhance phatic communion and expres-
sive speech acts, especially greetings’ (p. 109). Focusing on how sarcasm is realised in
written communication, Thompson and Filik (2016) observe that ‘tongue face, wink
face, and ellipsis all occurred significantly more frequently with sarcastic than literal
comments’ (p. 116). Similarly, Skovholt et al. (2014) observe that emoticons are used as
markers of humorous intent and irony, and that they can be used to hedge accompanying
4 new media & society 00(0)

speech acts, such as face-threatening and directive acts. Another avenue of pragmatic
enquiry has investigated how emoji and emoticons can be used as discourse markers akin
to punctuation, but that can simultaneously realise other kinds of meaning (c.f. Dresner
and Herring, 2010; Na’aman et al., 2017; Provine et al., 2007; Sampietro, 2016). These
findings are confirmed by work on emoji within the field of grapholinguistics (Dürscheid
and Siever, 2017).
Danesi’s (2016) work marshals various strands of phatic, pragmatic and semantic
research to address the question of whether emoji constitute an independent language,
ultimately concluding that they do not (yet). Danesi distinguishes adjunctive emoji that
make meaning alongside language, and substitutive emoji, that replace language, and
observes that emoji-intelligibility declines in correlation to its independence from lan-
guage. Danesi also emphasises that unlike language, emoji interpretation does not require
any formal instruction. In describing emoji semiosis, Danesi foregrounds their ‘adding
tone’ function, which is congruent with the ‘tone modification’ function described by
Herring and Dainas (2017).

Context-sensitive approaches
While work exploring how emoji make meaning independently of language has yielded
valuable insights, with sustained, widespread use, emojis’ specific pragmatic meanings
may have become diluted (Konrad et al., 2020). In response, some scholars have adopted
analytical frameworks that aim to describe how emoji make meaning in combination
with linguistic resources. In this vein, grapholinguistic research on emoji advocates con-
text-specific interpretation of their meaning (Dürscheid and Meletis, 2019). Accordingly,
Dürscheid and Meletis (2019) state, ‘this renders the emoji graphematically ambiguous,
as the specific linguistic unit it refers to is not fixed but variable and determined by the
context or the reader’s interpretation of a given text in which it is used’ (p. 174). Similarly,
accommodating co-occurring language into their analytical framework, Ge and Herring
(2018) adapt rhetorical structure theory to analyse how emoji sequences relate to their
accompanying co-text rhetorically and logically, concluding that in some respects, emoji
appear to be ‘developing into an independent language’ (p. 15).
A related vein of argument that runs through analysis of emoji and emoticons is the
likening of these resources to gesture. At a surface level, this is motivated by the popular-
ity of emoji which iconically represent facial expressions and body gestures (i.e. ‘Thumbs
Up’ , ‘OK Hand’ ). Describing the relationship between emoji and gesture, Dresner
and Herring (2014) comment that their analysis ‘does not rule out an iconic mapping
between the function of emoticons and some bodily and facial movements’ (p. 66).
However, a direct equivalence of emoji and gesture risks proscription of emoji’s meaning
potential – as Albert (2020) observes, ‘the formal analogy between emoji faces in general
and the corresponding facial expressions provokes the misleading inference that there
must also be a functional analogy’ (p. 68). A more nuanced parallel drawn between these
modes is that like co-speech gesture, the meaning made by emoji is to varying degrees
dependent on their linguistic co-text. This dependent relationship to language prompts
Abercrombie’s (1968) designation of modes such as co-speech gesture as ‘paralinguistic’,
a term we adopt here. Employing McNeill’s (1992) diagnostic criteria for determining the
Logi and Zappavigna 5

degree to which semiotic modes can function independently of language, Gawne and
McCulloch (2019) observe that ‘gestures and co-speech emoji are closely integrated into
meaning with the accompanying speech/text’ (p. 13). They argue that unlike language,
emoji are global and synthetic, non-combinatoric, context-sensitive and do not have
standards of form. In so doing, they integrate the pragmatic, phatic or attitudinal functions
of emoji found in earlier work into a single model for emoji semiosis.
In treating emoji as a paralinguistic modality and allowing interpretation of their
meaning to be guided by co-text, Gawne and McCulloch’s work aligns with a social
semiotic approach such as that adopted by Zappavigna (2012, 2018) and Parkwell (2019).
This approach maps how emoji realise meaning across the three metafunctions described
in the social semiotic theory of SFL. Accordingly, Zappavigna’s (2012) analysis of
emoticons used in Twitter posts described their function of construing (generally posi-
tive) interpersonal meaning among interactants, but also noted their textual function as
discourse markers. In line with social semiotic work on intermodal semiosis (c.f. Martin,
2008), Zappavigna’s (2012) analysis interpreted emoticons in context, observing that
‘Viewed alone, emoticons display a high degree of “fuzziness” . . . interpersonal mean-
ing is more readily studied when we look at how emoticons work in tandem with evalu-
ative meanings made in the verbiage’ (p. 80). A further consequence of this ‘fuzziness’ is
that meaning construed by emoticons cannot be described by more delicately differenti-
ated features of semiotic systems. Whereas in SFL theory linguistic resources realising
attitude2 can be subdivided into affect, appreciation and judgement, Zappavigna (2012)
found that viewed in isolation, emoticons can only construe generalised attitude.
Moreover, with regard to distinguishing how emoticons construe solidarity and attitude,
Zappavigna, (2012) notes that ‘it is not possible to neatly compartmentalize emotion and
solidarity. Emotion is involved in the way solidarity is expressed since alignment with
others is generally a positive emotional experience’ (p. 68).
Also approaching emoji semiosis from a social semiotic perspective, Parkwell’s
(2019) analysis elaborated a framework for describing how meaning is made across
modes. Parkwell drew on social semiotic work on multimodality (Kress and Van
Leeuwen, 2001; O’Halloran, 2004), and on Zappavigna’s (2018) analysis of the dis-
course of social media, in particular, their metafunctional analysis of hashtags. Parkwell’s
(2019) analysis explored how a single emoji (‘toilet’ ) can construe meaning in each of
the metafunctions, concluding that ‘emoji are a highly contextual, flexible modality,
likely to continue to shift and morph with the changing needs and contexts of social
media users’ (p. 9).

Systemic Functional Linguistics (SFL)

A social semiotic approach to exploring emoji
As attested by the work of Zappavigna (2012, 2018) and Parkwell (2019), a social semi-
otic approach to exploring emoji offers a robust analytical framework for describing
intermodal semiosis. This framework is underpinned by the theoretical architecture of
SFL as elaborated by Halliday (1978) and colleagues. A key theoretical principle of SFL
relevant to this study is the complementarity of paradigmatic and syntagmatic relations
as perspectives on semiosis. Language users make meaning by instantiating choices
6 new media & society 00(0)

from systems of language, and these choices interact as a text unfolds. Paradigmatic rela-
tions describe the relationship between all the potential options available to a language
user at a particular point in a text, or as Halliday and Matthiessen (2014) indicate, ‘what
could go instead of what’ (p. 22). Syntagmatic relations, however, are concerned with
‘what goes together with what’ (Halliday and Matthiessen, 2014: 22), offering insight
into the patterns and regularities that occur as language unfolds. In this research, into the
semiotic affordances of emoji, paradigmatic relations provide a view on how emoji are
organised into systems of options, while syntagmatic relations account for their contribu-
tion to a text’s meaning as it unfolds as discourse.
Another key dimension of SFL useful for understanding emoji is the concept of semi-
otic metafunctions, whereby semiotic resources simultaneously enact an ideational func-
tion of construing experience, an interpersonal function of enacting relationships and a
textual function of organising discourse into coherent text. The choices in meaning most
relevant to this study are at the level of discourse semantics, concerning the ideational
system, ideation, the interpersonal systems, appraisal and involvement, and the textual
system, periodicity. These systems are explained in more detail in the ‘Dataset and
method’ section where they form the basis of the coding rubric applied to annotate the
dataset.

Intermodal semiosis
Within the social semiotic tradition, multimodal discourse analysis (MDA) has emerged
as a robust framework for analysing how non-linguistic modes realise meaning, both
independently and in conjunction with language (Kress and Van Leeuwen, 2001). MDA
is underpinned by the SFL theoretical architecture described earlier. Accordingly, multi-
modal discourse analysts have elaborated systems that describe how paradigmatic and
syntagmatic choices unfold in modalities other than language, and how resources in
these modes realise meaning across metafunctions. A further, crucial region of enquiry of
MDA is how meaning enacted across modalities interacts, given that many genres
employ multiple modalities within a single text. This approach is informed by what
Lemke (1998) terms the ‘combinatorial principle’ for interpreting intermodal semiosis,
which ‘shows us how we can mean more, mean new kinds of meanings never before
meant and not otherwise mean-able, when this process occurs both within and across
different semiotic modalities’ (p. 92). Lemke’s approach to describing intermodality thus
proposes that the most fruitful analysis of intermodal texts accounts for both meaning
made within modes and across them. The term ‘intermodality’ is favoured over ‘multi-
modality’ here as it foregrounds the relation among modes as the site of enquiry, rather
than the description of their semiosis in parallel (see Bateman, 2014). Reflecting on work
exploring the interaction among linguistic, paralinguistic and pictorial resources in peda-
gogic contexts, Lemke (1998) notes that despite the analytical utility of separating these
‘into different “channels”’, this approach ‘neglects the inherent unity of communicative
meaning-making which makes the co-ordination among channels not only possible, but
normal’ (p. 94). Accordingly, while the method adopted here begins by analysing linguis-
tic and emoji semiosis monomodally, description of meanings construed across texts
reflects a unitary interpretation. In this regard, the MDA approach parallels the
Logi and Zappavigna 7

context-sensitive approaches described in the previous section, such as the rhetorical

structure theory-informed approach employed by Ge and Herring (2018) and the grapho-
linguistic approach taken by Dürscheid and Meletis (2019).
As the combinatorial principle has been applied to analyses of intermodality in other
contexts and genres, a number of more specific intermodal relationships have been
observed. Those most relevant to this study are intermodal coupling and minimum map-
ping. A social semiotic multimodal discourse analysis approach is also relevant to the
polemic regarding whether emoji can be considered a stand-alone language (c.f. Danesi,
2016). This approach sidesteps the question by considering emoji a semiotic resource
(like images, sounds, etc.) with particular semiotic affordances. As such, while we refer
to SFL’s ‘supramodal’ grammar to describe these affordances, we recognise that in some
regions of meaning-making emoji afford fewer and thus less delicate options for making
meaning than language, but in others they afford more.

Intermodal coupling.  Intermodal coupling provides the core unit of analysis for describing
intermodal semiosis. An intermodal coupling occurs when resources across communica-
tive modes interact, yielding a new semiotic unit. This definition follows the approach
outlined by Painter et al. (2013). While Painter et al.’s work was limited to interactions
within metafunctions (constrained by the principle of convergence/divergence), they
acknowledge that intermodal couplings can also occur across metafunctions. Work on
how resources interact across metafunctions (but within a single modality) has been
especially productive in describing the interaction of attitudinal and ideational resources
(e.g.: Knight, 2013; Martin et al, 2013; Zappavigna and Martin, 2018). These resources
frequently interact to construe language users’ evaluations about ideational targets; as
Martin (2004) states, ‘feelings are always about something – they are always interper-
sonal attitudes to ideational experience’ (p. 337). For example, in the linguistic compo-
nent of text #77 in the dataset for this study,

Best day of my life so far

the positive attitude (appreciation) Best is targeting the ideational entity day of my life
so far. Consequently, we would describe the resulting meaning as an evaluative coupling,
notated (following Martin et al, 2013) as (day of my life so far/+ve appreciation) (‘+ve’,
‘–ve’ are used as shorthand for ‘positive’, ‘negative’; ‘appreciation’ is the subcategory of
attitude describing resources for construing aesthetic evaluations). Accounting for the
role of emoji resources in this text introduces an additional layer of complexity, as the
positive attitude realised by the emoji interacts with the positive appreciation of Best,
thus also participating in the evaluative coupling. A principled description of this interac-
tion requires us to account for how meaning is distributed across these modes, as well as
how their individual meanings combine within the coupling. In response, we incorporate
the principle of minimum mapping into our analytical framework.

Minimum mapping.  Complementing Painter et al.’s work, Zhao (2011) proposes the mini-
mum mapping hypothesis as part of a more principled framework for the analysis of
meaning across modes. This hypothesis states that ‘if verbiage and image can
8 new media & society 00(0)

co-construe one aspect of a social action, e.g. process (what is going on), participant (the
participants that engage in the process), etc., they form a verbiage-image coupling’
(Zhao, 2011: 171). Thus, while individual resources might potentially construe general-
ised or multiplied meanings, when they co-occur in a text their meaning potential is
constrained to the region of meaning shared among them. See, for instance, text #44:

Incheon Airport

Here, the emoji in isolation does not necessarily denote the ‘airport’ entity (or rather,
it is a less salient option of its meaning potential); however, when combined with the
lexical item ‘airport’ as a coupling, the meaning across these modes is specified as con-
struing that entity (c.f. Dainas and Herring’s (2021) ‘mention’ function of graphicons).

Dataset and method

The dataset considered in this study is a corpus of 1000 digital messages (text messages
and social media posts) collected in 2019 and containing at least one emoji. The dataset
was collected from 50 undergraduate media students at an Australian University, with
each student contributing 20 messages. Each of these students was interviewed for 1 hour
about the emoji they used in these messages. Due to the scope of the study and the neces-
sity for close textual analysis of both the emoji and the co-text across multiple dimensions
of meaning, a random selection of 200 texts was analysed. Of this selection, approxi-
mately 40 (see Appendix 1) are discussed in this study. The decision of which to include
was guided by the methodological principle of theoretical sampling as proposed by Glaser
and Strauss (2006), whereby an ‘analyst jointly collects, codes, and analyzes his data and
decides what data to collect next and where to find them, in order to develop his theory as
it emerges’ (p.45). Accordingly, the examples presented in this article were selected for
their value in illustrating particular theoretical phenomena noted by the analysts.
The method for analysing the data was qualitative, with individual messages manu-
ally annotated according to a coding rubric. This rubric was designed to code for the
kinds of meanings that emoji can make intramodally (independent of language) and
intermodally through various kinds of relations with the linguistic component of the
message. Categories of meaning were drawn from Martin (1992) and Martin and White’s
(2005) discourse semantics systems of ideation (construing experience), appraisal
(expressing emotion and evaluation), involvement (enacting solidarity), and periodicity
(organising text). Linguistic realisations of features within these systems are shown in
Table 1. Relationships between individual features were annotated as couplings, drawing
on the relational principles of intermodal semiosis explained earlier in this article.
Accordingly, the following four kinds of intermodal couplings were identified: conver-
gent ideation, convergent attitude, convergent graduation, and intermodal evaluative
couplings. By coding the meaning realised by emoji first in isolation and then in con-
junction with linguistic co-text, this method allowed the researchers to identify instances
where the interaction between modalities impacted emoji meaning. In turn, these
instances form the basis of the results and discussion.
The annotation software, WebAnno, was employed to systematically code the data.
WebAnno is a Java-based, web-hosted annotation programme that accommodates
Logi and Zappavigna 9

Table 1.  Coding Rubric for discourse semantic systems for annotation of layers in WebAnno.

System Subsystem Description Example

ideation entity Things, activities and coffee (#10 from dataset),
semiotic entities helping (#147), publication
(#52).
occurrence Activities or processes they’ve taken it off Netflix
involving one or more (#64), I had salad (#63).
entity
state States or changes Hyundai exec for Sale
affecting one or more (#31).
entities
appraisal attitude Emotion or evaluation affect (emotion):
happy (#183), sad (#7);
judgement (social esteem/
sanction): wholesome
(#181), freaks (#200).
graduation Up or downscales cutest (#90).
attitudinal resources
involvement Non-gradable resources twinning (#45) (slang term
for construing solidarity for resembling another
among interactants, person’s appearance).
including resources that
mark interactants as part
of a shared community,
such as swearing or taboo
lexis, slang and naming
periodicity Organisation of text in sometimes you gotta
terms of information let your skin breathe a
structure little (#200) (temporal
circumstance in Theme
position).

Unicode characters such as emoji and has a flexible, customisable interface for designing
coding rubrics. Data were first converted to .csv file format and uploaded into a new
WebAnno project, formatted for one entry per line. The software enabled two kinds of
annotation to occur: annotation of features in discrete layers (coding meaning for iso-
lated semiotic resources, such as a single emoji or lexical item) and annotation of types
of relations between features. The discourse semantic systems shown in Table 1 formed
the annotation ‘Layers’ within WebAnno, forming ‘tags’ with ‘tagsets’.
An example of how these layer features and couplings are shown when coded in the
WebAnno interface is found in Figure 1 – an excerpt of the coding for text #13. We can
see that each individual feature and coupling is labelled in the interface and, while it can-
not be shown in a static image such as this one, each label is highlighted in bold font
when the mouse pointer is held over it (like ‘cumulative’ in this image). The programme
10 new media & society 00(0)

Figure 1.  Example of WebAnno coding.

abbreviates the features in the visualisation: Att = Attitude; Ide = Ideation; Pos =
Position (relating to the location in the text where an emoji occurs); Pol = Polarity
(specifying whether instances of attitude are positive or negative). Coding of emoji
resources for these features was conducted by interpreting emoji intramodally (within a
single mode) and then intermodally in batches of 40–50 texts. After each batch, the
researchers would confer to discuss any issues in the coding rubric or points of interest.
In light of revisions to the coding rubric or how it should be applied to particular cases,
coding was repeated or updated where necessary.
Considering the importance of the context of communication for interpreting the mean-
ing of emoji, the authors of texts were interviewed and asked about the tenor of the text (the
relationship between the author and recipient or audience), the field of the text (the topic or
domain), and the mode (the platform through which the text was produced and dissemi-
nated). Annotation results were compared to these descriptions. The degree of convergence
(agreement) between the annotation and the authors’ descriptions were coded as either ‘fully
agree’, ‘partially agree’ or ‘disagree’. The annotation fully agreed with the author descrip-
tions in 81% of cases, and if including partial agreement, accorded with 93% of instances.

Results and discussion

Given the goal of mapping emoji semiosis according to SFL theory, a primary outcome
of the analysis was the identification of typical emoji realisations of semiotic choices in
SFL discourse semantic systems. Once this had been concluded, these realisations
informed analysis of more complex instances of intermodal semiosis. Accordingly, the
results of this study can be divided between the elaboration of a social semiotic analytical
framework for analysing emoji semiosis and the description of particular regions of
intermodal meaning realised by the interaction between emoji and language.

Typical emoji realisation of discourse semantic features

The first relevant result of the analysis is the description of typical meanings realised by
emoji. Identification of typical emoji realisations of discourse semantic features was
guided by an understanding of how intermodal semiosis acts to foreground particular
Logi and Zappavigna 11

Table 2.  Attitude realised by emoji.

Layer Description Examples

Positive polarity Negative polarity

attitude Facial expressions

Emblematic gestures ️

Ideational entities/
occurences/states

Table 3.  graduation realised by emoji.

Layer Description Examples

graduation: Repetition
upscale
Intensification of facial ‘grinning face with big eyes’ and ‘weary
expression face’ .
Iconic representation of 5 places left! There are only FIVE
paralinguistic emphasis tickets remaining (#40)

regions of emoji meaning potential. We are aware that capturing the dynamic, context-
dependent semiosis of emoji in a static representation such as a table inevitably simpli-
fies and omits aspects of individual emojis’ meaning potential; however, given the
repeated use of some emoji to construe particular semiotic features we feel it is justified
to note these typical realisations. These realisations are summarised below and form the
basis for the descriptions of semiosis realised through the interaction between emoji and
language discussed in the following subsections.
attitude is the discourse semantic system within the interpersonal metafunction con-
cerned with describing resources for construing evaluative meaning. Typical emoji realisa-
tions of attitude include iconic representations of facial expressions, emblematic gestures
and ideational entities or occurrences or states with attitudinal connotations (Table 2).
graduation is the discourse semantic system for upscaling or downscaling attitude.
In the data analysed here, emoji appear to only realise upscaling of attitude. This is
achieved either through repetition of emoji construing attitude, through emoji realising
stylised icons of intensified facial expressions (as compared to a baseline realisation such
as the Smiling Face ), or through iconic realisation of ideational entities that connote
paralinguistic features that realise upscaled emphasis (Table 3). An example of the latter
12 new media & society 00(0)

Table 4.  involvement realised by emoji.

Layer Description Examples

involvement Community ‘rainbow’ encoding solidarity among members of
specific emblems the lgbtq+ community in text #12:
Love is love Proud of love.
Taboo semiosis ‘Pile of Poo’ emoji encoding positive attitude among
interactants in text #17:
Happy birthday to my fave twins!!!!
#thosewhogetfattogetherstaytogether

Table 5.  ideation realised by emoji.

Layer Description Examples

entity occurrence
ideation Iconic representation ‘desert island’ ‘person running’
of entities and (#11) (#40)
processes

option is found in text #40, where the raised volume in spoken language connoted by the
‘loudspeaker’ emoji serves to upscale the written language in the text.
involvement is the interpersonal system for expressing solidarity by employing semi-
otic resources to encode meaning that can only be interpreted by members of a particular
community. In the context of emoji semiosis, these include emoji that are used as
emblems within particular communities and emoji that connote taboo meaning which is
acceptable when used among solidary interactants (Table 4).
ideation is the discourse semantic system describing resources for construing experi-
ence. Emoji typically encode ideational meaning through iconic representations of phys-
ical entities, processes and qualities (Table 5). While emoji were observed to realise
entities and occurrences even independently of language, all instances where emoji real-
ise state figures occurred as intermodal couplings.
The periodicity system describes choices in how information is ordered within a text.
As emoji are less formally bound by syntagmatic relations to surrounding co-text than
linguistic resources, the coding rubric for periodicity choices has been simplified to three
features corresponding to emoji location within a text: initiating (occurring before lin-
guistic resources), integrated (occurring after some and before other linguistic resources),
or cumulative (occurring after linguistic resources; Table 6).

Annotation of relations between features as couplings

The second relevant result of the analysis concerns the categorisation of coupling rela-
tions found in the data. Couplings among semiotic resources can occur either within or
across metafunctions and modalities, thus coupling options can be represented as a
Logi and Zappavigna 13

Table 6.  periodicity realised by emoji.

Layer Description Examples

Initial Integrated Cumulative

periodicity Where the We are finalists chillssssss do we really
emoji is in the category of @ have to do
located in Fitness Service in 2019 username(#73) this (#22)
the text. Melbourne North Local
Business Awards! (#89)

Table 7.  Applying a semiotic couplings matrix to text #90.

Intrametafunctional
Intramodal Coupling of interpersonal
metafunction resources for attitude
(cutest) and graduation (cutest)
within the linguistic mode of text
#90.
Intermodal Coupling of interpersonal
metafunction resources for attitude
in the linguistic mode (cutest) and in
emoji ( ) in text #90.
Intermetafunctional
Coupling of interpersonal
metafunction resources for attitude
(cutest) and ideational (baby) within
the linguistic mode of text #90.
Coupling of interpersonal metafunction
resource of attitude in the linguistic
mode (cutest) with the ideational
metafunction resource of ideation in
the emoji mode (‘Cat Face’ )

matrix organised around two variables: intramodal versus intermodal and intram-
etafunctional versus intermetafunctional, as shown in Table 7 referring to text #90:

the cutest baby

Describing intermodal semiosis within a metafunction is substantially different to

describing intermodal semiosis across metafunctions, thus we have divided intermodal
couplings into two kinds: convergent intermodal couplings and evaluative intermodal
couplings. In turn, convergent intermodal couplings can be further specified by metafunc-
tion or system, for instance, ideational convergent intermodal coupling or attitudinal
convergent intermodal coupling (illustrated in Table 8).
However, it should be noted here that in instances where emoji and language con-
verge to construe attitude, that attitude will always have an ideational target, thus an
evaluative coupling will also be construed. This is the case in text #56:

The good stuff

Here, emoji and language converge to construe positive appreciation for the idea-
tional target stuff, thus forming the evaluative coupling (stuff/+ve appreciation). As
posited by the minimum mapping hypothesis, the intermodal relationship exists between
14 new media & society 00(0)

Table 8.  Discourse semantic features realised by intermodal couplings.

Feature Subtype Realisation examples Description

Convergent entity Leighton Beach (#43) Leighton Beach converges with
ideation ‘beach’
occurrence Do you want to today? ‘flexing biceps’ converges
(#188) with the mental process
clause do you want to to
construe the occurrence ‘lift
weights’
state new nose piercing (#192) ‘sparkles’ converges with
nose piercing to construe the
state ‘sparkling nose piercing’
Convergent affect its sad bc this is the standard sad converges with
attitude for literally 90% of the media/ ‘persevering face’
pr/marketing internships i see
online (#7)
appreciation The good stuff ((#56) Interaction between good and
‘Ok hand’ :
judgement Went to harry styles concert what a guy converges with
with 0 expectations but came ‘Star-Struck’
back in awe???? What a guy
(#72)
Convergent graduation Youre such a little such converges with
graduation (#9) repetition of ‘mouse’
Evaluative Is anyone selling Taylor Swift Is anyone selling Taylor Swift
coupling tickets? . (#176) tickets? realises an ideational
occurrence, while realises
negative attitude targeting
this occurrence.

the meaning construed by each mode as well as between attitudinal and

attitudinal
ideational meaning divided by mode, thus this text contains both a convergent attitudi-
nal coupling (good + ) and an evaluative coupling (stuff + ).

Applying principles of minimum mapping to the dataset

The third relevant result of the analysis concerns the application of the minimum map-
ping hypothesis to the interpretation of meaning made by the interaction between emoji
and language. Application of the principle of minimum mapping introduced in Section
‘Minimum mapping’ to emoji-language semiosis revealed that the semiotic relations
between interacting resources are temporary. Thus, individual resources could act to
specify differing aspects of their meaning potential depending on what they were
Logi and Zappavigna 15

coupled with, the coupling’s co-text, and the text’s context. For example, the emoji
was used in texts #56 and #88:

#56: The good stuff

#88: Take 3 for the sea (more is welcome too :);) ).

In text #56, the emoji’s interaction with the verbiage good foregrounds its attitudinal
meaning (as the emoji iconically represents the hand gesture emblematic for ‘OK’), with
the resulting intermodal coupling construing positive appreciation. Contrastingly, in text
#88, interaction with the verbiage three foregrounds the ideational aspect of the emoji
realising the entity ‘three’ (as the emoji iconically represents a hand holding up three
fingers). In this dataset, no instances were found of individual resources being repeated
within a text with divergent meanings foregrounded, most likely due to the short length
of texts. However, the underlying principle of temporary interaction allows for such
divergence, especially in longer texts.
Minimum mapping was useful in guiding analysis of interaction across modes within
a single metafunction, but it did not help us describe intermodal couplings where each
mode is realising meaning in a different metafunction. Take, for example, text #176:

Is anyone selling Taylor Swift tickets? .

In this text, ideational meaning is realised by the linguistic component of the text, while
attitudinal resources are realised by the emoji component. We can interpret the emoji as
construing generalised negative attitude, and the field of the ideational meaning is
established as ‘obtaining tickets to a Taylor Swift concert’. But with no interacting
resources within the interpersonal metafunction to minimally map onto, the attitude of
the intermodal evaluative coupling remains unspecified (c.f. Zappavigna’s (2012)
description of ‘fuzzy’ meaning). Plausible interpretations of the emoji include negative
affect, reflecting the author’s concern about finding a ticket, or negative appreciation for
the absence of tickets. Consequently, in cases such as this, we refrain from specifying
more delicate attitude features and notate the coupling as (obtaining tickets to a Taylor
Swift concert/–ve attitude).

Taxonomic relations within convergent ideation entity couplings

The fourth relevant result of the analysis concerns the distribution of meaning across
modes in intermodal, convergent ideational couplings. A particular region of intermodal
semiosis that was observed in the data is that convergent ideational meaning realising
entities across modes often follows a pattern of hypo or hypernymy, whereby meaning
construed by one mode will be a superordinate or subordinate to meaning construed by
the other. For example, text #55:

Thoroughly disappointed with the lack of milk

16 new media & society 00(0)

Figure 2.  Intermodal ideational semiosis in text #55.

Here, the verbiage milk realises one primary entity – ‘milk’ (other potential interpre-
tations are possible, as [etc.] in Figure 2 allows for). Conversely, the ‘baby bottle’ entity
iconically realised by the emoji can be divided into the entities that constitute the
graphicon: ‘bottle’, ‘milk’, ‘teat’ and so forth. Applying the minimum mapping hypoth-
esis, we propose that the intermodal meaning jointly construed by the emoji and lan-
guage is the ‘milk’ entity construed by both modes, while other entities potentially
construed by only one mode are not realised. The intermodal semiosis of this text is
summarised in Figure 2, with linguistic semiotic resources highlighted in red, emoji in
blue and intermodal semiosis in green. In turn, the intermodally construed entity ‘milk’
is nested within a linguistically realised state, ‘lack of milk’.
A more complex example of shared ideational meaning mapped across modes can be
seen in text #198:

If you want a break from studying and pet some animals there is a petting zoo at the Physics
Lawn (in front of Parker apartments) RIGHT NOW until 1pm !!! It’s entirely free and there are
trained handlers with lots of animals

Here, we can see how the linguistically construed ideational entities some animals,
petting zoo, trained handlers and lots of animals can be organised into a taxonomy that
intersects with the meaning of the ‘cow’ , ‘rabbit’ and ‘goat’ emojis, as shown
in Figure 3. Collectively, language and emoji realise a taxonomic category coded as ‘pet-
ting zoo animals’. This relationship echoes Zhao’s (2011) description of ‘metonomysing’
relations in language-image texts.

Intermodal specification
A fifth noteworthy pattern observed in the data is that intermodal interaction can serve to
specify particular dimensions of the meaning potential of each mode. Consider, for
example, text #169:
Logi and Zappavigna 17

Figure 3.  Intermodal ideational semiosis in text #198.

Smaller than expected but cosy none the less (: Saw . I reckon it would be a
whole ‘nother scene at night. Wouldn’t mind coming back in spring sometime as well with all
the flowers in bloom~

Here, we can see how Saw construes an intermodal occurrence which can
be rendered linguistically as ‘I saw pigs, chickens and goats’. However, following from the
syntactical patterning of the preceding co-text, rather than construing its typical, mono-
modal meaning of ‘knife and fork’ or ‘cutlery’, the emoji construes the process ‘eating’
as part of the occurrence which can be rendered as ‘ate rabbit’: . This interpretation
also sustains continuity in the ideational field of the text, with farm animals remaining the
targets of the author’s activities (as opposed to incongruously suggesting that the author
saw pigs, chickens, goats, cutlery and rabbits). From a minimum mapping perspective, the
verbal group dimension of the emoji has been foregrounded because it intersects with the
meaning realised by the co-text, while the entity dimension is backgrounded.
Another instance of linguistic co-text shifting the meaning of emoji from an idea-
tional entity to an occurrence occurs in text #67:

I can see how ‘interested’ everyone is

Here, the quotation marks around ‘interested’ suggest this word could be interpreted
ironically or sarcastically, implying that ‘everyone’ is in fact not interested. This opens a
potential region of meaning with which the emoji which follows can be cross-refer-
enced, inviting a reader to consider what relationship might exist between the state ‘eve-
ryone is (not) interested’ and the iconically realised entity ‘mobile phone’. Accordingly,
a plausible interpretation of the emoji sequence is that it construes an
18 new media & society 00(0)

Figure 4.  Intermodal ideational semiosis in text #67.

occurrence rendered as ‘looking at mobile phones’ which in many contexts is associated

with lack of interest. In this case, it is noteworthy that the interaction among modes func-
tions to specify meaning bi-directionally, as both the quoted verbiage, ‘interested’ and
the emoji are examples of semiotic resources that are to some degree indeterminate
and context dependent. By applying a minimum mapping approach to interpreting inter-
modal semiosis, we can see how the dimensions of individual semiotic resources that are
shared across modes are foregrounded, while those that are not shared are backgrounded.
The intermodal semiosis occurring in this text is summarised in Figure 4.

Proximal attitudinal prosody

A final observed pattern in attitudinal meaning mapped across modes is that attitude
realised by emoji typically ‘washes’ over adjacent co-text both forwards and back-
wards. As such, we have termed this proximal attitudinal prosody. As Halliday (1979)
and Martin (1995, 1996) have described, different kinds of meaning unfold according
to different structural configurations. Halliday (1979) describes interpersonal meaning
as ‘strung throughout the clause as a continuous motif or colouring’ whose ‘effect is
cumulative’ (p. 67). For example, in text #27, we can see how the positive attitude
construed by the words Merry, safe and happy accumulates and intensifies over the
course of the text:

A belated Merry Christmas and a safe and happy holidays to y’all

It is thus unsurprising to see attitude construed by emoji contributing to prosodic

meaning, for instance, in text #52:

Extremely chuffed that my first publication, an article for Discourse & Society, is now online
– a nice wrap on the first fortnight of my PhD

Here, the general positive attitude construed by ‘Grinning Face with Big Eyes’
builds on the positive attitude realised by chuffed and nice, combining to realise
Logi and Zappavigna 19

convergent attitude couplings with these lexical items of affect and appreciation,
respectively.
A more unexpected observation in the interaction between intermodal attitude
resources is that while the prosodic unfolding of attitudinal meaning in the linguistic
mode is largely unidirectional, attitudinal emoji appear to demonstrate a freer relation to
their co-text. An example of this can be found in text #99:

Me & My worst frenemy.

In this text, the first attitude resource is the lexical item worst, which construes
upscaled negative appreciation. This is followed by frenemy – a portmanteau of the lexi-
cal items ‘friend’ and ‘enemy’ – which denotes both positive and negative affect.
Looking at only the linguistic resources, the attitudinal prosody of this text would appear
to be either negative or ambivalent. However, interaction with the positive attitude con-
strued by the ‘Red Heart’ emoji foregrounds the positive attitude of the verbiage,
while backgrounding the negative, resulting in an overall attitude construed in the text
being positive. This suggests that the attitude construed by the emoji is ‘washing’ back-
wards over the preceding co-text in a kind of reverse prosody. This aspect of convergent
attitudinal couplings parallels the ‘tone modification’ function observed in pragmatics-
informed descriptions of emoji semiosis such as Herring and Dainas (2017).
While the relation of emoji to attitudinal prosody does appear to be freer than that of
linguistic resources, attitudinal emoji can nonetheless be governed by syntagmatic rela-
tions within a text. For instance, in text #111, the ‘Crying Face’ and ‘Red Heart’
emoji only interact with the clauses that precede them, acting as culminations for the
linguistic attitude construed therein:

Gonna miss this school Love every one of you boys, thanks for the memories

This suggests attitudinal meaning realised by emoji behaves somewhat differently

from linguistic attitude, both in how it interacts with proximal attitudinal resources and
how it can serve to conclude stretches of prosody. This observation parallels work both
within social semiotics (such as Knox, 2009) and in other theoretical traditions that notes
how emoji and emoticons can function like clause-final punctuation (c.f. Dresner and
Herring, 2010; Na’aman et al., 2017; Provine et al., 2007; Sampietro, 2016).

Conclusion
This article has presented a social semiotic perspective on emoji semiosis. Informed by
the theoretical architecture of Systemic Functional Linguistics (SFL) and social semiotic
work on Mulitmodal Discourse Analysis (MDA), we have proposed an analytical frame-
work that describes how emoji and language interact as semiotic resources for making
meaning. This framework was applied to the data to map typical emoji realisations of
discourse semantic choices across ideational, interpersonal and textual metafunctions,
revealing patterns in how meaning is construed across modalities. It was observed that
intermodally convergent ideational meaning can be described as a shared ideational
20 new media & society 00(0)

taxonomy wherein meaning realised in one modality is sub- or superordinate to the other,
thus construing an intermodal field of discourse. A second noteworthy interaction
between emoji and language was revealed by a minimum mapping approach to intermo-
dality: the region of meaning potential that overlaps across modes corresponds with the
region of meaning realised by each mode. This supports the productivity of an MDA-
informed approach to interpreting emoji semiosis and provides a valuable guiding prin-
ciple for emoji-language intermodality. A third pattern observed in the interaction
between emoji and language is that the attitude realised by emoji behaves somewhat
differently from linguistically construed attitude insofar as it appears to interact prosodi-
cally with proximal linguistic resources occurring both before and after in the co-text.
These results constitute a modest advancement towards mapping the relations govern-
ing how emoji and language jointly construe meaning. They take us beyond the correct
but vague observations regarding the fuzziness and context-dependency of emoji preva-
lent in the area of study to date, and offer the potential for a more principled study of
these relations in future. It might be especially fruitful to combine the social semiotic
approach of this study with quantitative approaches in Artificial Intelligence (c.f. Miller
et al., 2016) and theories of communication (c.f. Veszelszki, 2017) to explore how these
relations unfold across larger datasets.
While the theoretical framework of SFL and MDA methods differ from more prev-
alent pragmatics-informed explorations of emoji semiosis, a number of points of
intersection between the two approaches were found. Methodologically, an MDA
approach to analysing how emoji and language interact to make meaning is congruent
with work informed by similarly intermodal frameworks such as the adapted rhetori-
cal structure theory employed by Ge and Herring (2018) and the grapholinguistic
approach of Dürscheid and Meletis (2019). In terms of results, convergent ideational
couplings appear to realise the ‘mention’ function described by Herring and Dainas
(2017), while the dynamics of proximal attitudinal prosody, where attitude realised by
emoji ‘washes’ across co-occurring language, largely corresponds to Danesi’s (2016)
‘adding tone’ function. These intersections both validate the findings of this work and
suggest the two traditions might be fruitfully cross-referenced in future research.
A salient limitation of this study is its limited dataset. Consequently, numerous semi-
otic functions observed by other researchers working with more varied data were not
observed in our data. For instance, it would be valuable to apply the analytical frame-
work proposed here to texts composed solely of emoji so as to explore how emoji con-
strue meaning in the absence of language. In light of recent work on emoji sequences
such as Ge and Herring (2018), this dimension of emoji use merits closer attention. A
further limitation of this study is that the data and method employed are unable to account
for the role the widely used emoji prediction features in digital keyboards play in sug-
gesting particular combinations of emoji and written language. Integrating results of
work in this area such as Barbieri et al. (2018) into the design of future research would
expand the descriptive power of resulting models.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/
or publication of this article: The research presented in this study was funded by the Commonwealth
government of Australia.
Logi and Zappavigna 21

ORCID iD
Lorenzo Logi https://orcid.org/0000-0001-9299-2307

Note
1. Emoji glosses are sourced from https://emojipedia.org/ (accessed 11 November 2020) which
collates the Unicode ‘Common Locale Data Repository (CLDR) Short Name’ for each emoji.
The CLDR project aims to provide locale data in an interoperable XML format so that it can
be used in a variety of computer applications.
2. Throughout this paper technical terms from Systemic Functional Linguistics are presented in
small caps to distinguish from their common-sense equivalents.

References
Abercrombie D (1968) Paralanguage. British Journal of Disorders of Communication 3(1): 55–59.
Albawardi AH (2018) Digital Literacy Practices of Saudi Female University Students. Reading:
University of Reading.
Albert G (2020) Beyond the binary: emoji as a challenge to the image-word distinction. In:
Thurlow C, Dürscheid C and Diémoz F (eds) Visualizing Digital Discourse. Berlin: De
Gruyter Mouton, pp. 65–80.
Barbieri F, Ballesteros M, Ronzano F, et al. (2018) Multimodal emoji prediction. arXiv preprint
arXiv:1803.02392.
Bateman JA (2014) Text and Image: A Critical Introduction to the Visual/verbal Divide. New
York: Routledge.
Crystal D (2001) Language and the Internet. Cambridge: Cambridge University Press.
Dainas A and Herring SC (2021) Interpreting emoji pragmatics. In: Xie C, Yus F and Haberland H
(eds) Internet Pragmatics: Theory and Practice. Amsterdam: John Benjamins.
Danesi M (2016) The Semiotics of Emoji: The Rise of Visual Language in the Age of the Internet.
London: Bloomsbury Publishing.
Dresner E and Herring SC (2010) Functions of the nonverbal in CMC: emoticons and illocutionary
force. Communication Theory 20(3): 249–268.
Dresner E and Herring SC (2014) Emoticons and illocutionary force. In: Riesenfeld D and
Scarafile G (eds) Perspectives on Theory of Controversies and the Ethics of Communication.
Dordrecht: Springer, pp. 81–90.
Dürscheid C and Meletis D (2019) Emojis: a grapholinguistic approach. In: Haralambous Y (ed.)
Graphemics in the 21st Century. Brest: Fluxus Editions, pp. 167–183.
Dürscheid C and Siever CM (2017) Jenseits des Alphabets – Kommunikation mit Emojis.
Zeitschrift für Germanistische Linguistik 45(2): 256–285.
Emogi Research Team (2016) 2016 emoji report. Available at: http://cdn.emogi.com/docs/
reports/2016_emoji_report.pdf (last accessed 26 July 2021).
Gawne L and McCulloch G (2019) Emoji as digital gestures. Language @ Internet 17(2). Available
at: https://www.languageatinternet.org/articles/2019/gawne
Ge J and Herring SC (2018) Communicative functions of emoji sequences on Sina Weibo. First
Monday. Available at: https://firstmonday.org/ojs/index.php/fm/article/view/9413
Glaser BG and Strauss AL (2006) Discovery of Grounded Theory: Strategies for Qualitative
Research. New York: Routledge.
Halliday MAK (1978) Language as Social Semiotic: The Social Interpretation of Language and
Meaning. London: Edward Arnold.
22 new media & society 00(0)

Halliday MAK (1979) Modes of meaning and modes of expression: types of grammatical struc-
ture, and their determination by different semantic functions. In: Allerton DJ, Carney E and
Holdcroft D (eds) Function Context in Linguistic Analysis: Essays Offered to William Haas,
vol. 1. Cambridge: Cambridge University Press, pp. 57–79.
Halliday MAK and Matthiessen CMIM (2014) An Introduction to Functional Grammar, vol. 4.
London: Edward Arnold.
Herring SC and Dainas AR (2017) ‘Nice picture comment!’ Graphicons in Facebook comment
threads. Paper presented at the proceedings of the 50th Hawaii international conference on
system sciences, Hilton Waikoloa Village, HI, 4–7 January.
Herring SC and Dainas AR (2020) Gender and age influences on interpretation of emoji functions.
ACM Transactions on Social Computing 3(2): 1–26.
Knight NK (2013) Evaluating experience in funny ways: how friends bond through conversational
hum. Text & Talk 33(4–5): 553.
Knox JS (2009) Punctuating the home page: image as language in an online newspaper. Discourse
& Communication 3(2): 145–172.
Konrad A, Herring SC and Choi D (2020) Sticker and emoji use in Facebook Messenger: implica-
tions for graphicon change. Journal of Computer-Mediated Communication 25(3): 217–235.
Kress GR and Van Leeuwen T (2001) Multimodal Discourse: The Modes and Media of
Contemporary Communication. London: Edward Arnold.
Lemke JL (1998) Multiplying meaning. In: Martin JR and Veel R (eds) Reading Science: Critical
and Functional Perspectives on Discourses of Science. Hove: Psychology Press, pp. 87–113.
McNeill D (1992) Hand and Mind: What Gestures Reveal about Thought. Chicago, IL: University
of Chicago Press.
Martin JR (1992) English Text: System and Structure. Philadelphia, PA: John Benjamins.
Martin JR (1995) Text and clause: Fractal resonance. Text-Interdisciplinary Journal for the Study
of Discourse 15(1): 5–42.
Martin JR (1996) Types of structure: deconstructing notions of constituency in clause and text.
In: Hovy EH and Scott DR (eds) Computational and Conversational Discourse. Berlin:
Springer, pp. 39–66.
Martin JR (2004) Mourning: how we get aligned. Discourse & Society 15(2–3): 321–344.
Martin JR (2008) Tenderness: realisation and instantiation in a Botswanan town. Paper presented
at the 34th international systemic functional congress, Syddansk University, Odense, 16–20
July 2007.
Martin JR and White PRR (2005) The Language of Evaluation: Appraisal in English. Basingstoke:
Palgrave Macmillan.
Martin JR, Zappavigna M, Dwyer P, et al. (2013) Users in uses of language: embodied identity in
Youth Justice Conferencing. Text & Talk 33(4–5): 467–496.
Miller H, Thebault-Spieker J, Chang S, et al. (2016) ‘Blissfully Happy’ or ‘Ready to Fight’: vary-
ing interpretations of emoji. Paper presented at the Proceedings of the international AAAI
conference on web and social media, Maternushaus, 17–20 May 2016, Vancouver, 30 July–4
August 2017.
Miltner KM (2021) ‘One part politics, one part technology, one part history’: racial representation
in the Unicode 7.0 emoji set. New Media & Society 23: 515–534.
Na’aman N, Provenza H and Montoya O (2017) Varying linguistic purposes of emoji in (Twitter)
context. Paper presented at the Proceedings of ACL 2017, Student Research Workshop.
Nishimura Y (2015) A sociolinguistic analysis of emoticon usage in Japanese blogs: variation by
age, gender, and topic. AoIR Selected Papers of Internet Research. Available at: https://spir.
aoir.org/ojs/index.php/spir/article/view/8396
O’Halloran K (2004) Multimodal Discourse Analysis: Systemic Functional Perspectives. London:
A&C Black.
Logi and Zappavigna 23

Painter C, Martin JR and Unsworth L (2013) Reading Visual Narratives. Sheffield: Equinox.
Parkwell C (2019) Emoji as social semiotic resources for meaning-making in discourse: mapping
the functions of the toilet emoji in Cher’s tweets about Donald Trump. Discourse, Context &
Media 30: 100307.
Provine RR, Spencer RJ and Mandell DL (2007) Emotional expression online: emoticons punctu-
ate website text messages. Journal of Language and Social Psychology 26(3): 299–307.
Rezabek L and Cochenour J (1998) Visual cues in computer-mediated communication: supple-
menting text with emoticons. Journal of Visual Literacy 18(2): 201–215.
Sampietro A (2016) Exploring the punctuating effect of emoji in Spanish WhatsApp chats.
Lenguas Modernas 47.
Skovholt K, Grønning A and Kankaanranta A (2014) The communicative functions of emoticons
in workplace e-mails. Journal of Computer-Mediated Communication 19(4): 780–797.
Thompson D and Filik R (2016) Sarcasm in written communication: emoticons are efficient mark-
ers of intention. Journal of Computer-Mediated Communication 21(2): 105–120.
Veszelszki Á (2017) Digilect: The Impact of Infocommunication Technology on Language. Berlin:
Walter de Gruyter GmbH & Co KG.
Walther JB and D’addario KP (2001) The impacts of emoticons on message interpretation in
computer-mediated communication. Social Science Computer Review 19(3): 324–347.
Zappavigna M (2012) Discourse of Twitter and Social Media: How We Use Language to Create
Affiliation on the Web. Bloomsbury.
Zappavigna M (2018) Searchable Talk: Hashtags and Social Media Metadiscourse. Bloombury.
Zappavigna M and Martin JR (2018) #Communing affiliation: social tagging as a resource for
aligning around values in social media. Discourse, Context & Media 22: 4–12.
Zhao S (2010) Intersemiotic relations as logogenetic patterns: the time factor in hypertext
description. In: Bednarek M and Martin JR (eds) New Discourse on Language: Functional
Perspectives on Multimodality, Identity, and Affiliation. Continuum, pp. 195–218.
Zhao S (2011) Learning through multimedia interaction: The construal of primary social sci-
ence knowledge in web-based digital learning materials. Dissertation/Thesis, Department of
Linguistics, University of Sydney, Australia.

Author biographies
Lorenzo Logi is a PhD candidate at the University of New South Wales, and is employing
Systemic Functional Linguistics to research the social semiosis occurring in stand-up comedy
performances. He completed his Master’s in Applied Linguistics at the University of Sydney in
2017, writing his dissertation on bonding and affiliation in television sitcoms. Recent publica-
tions include Logi, L. and M. Zappavigna (forthcoming). ‘Impersonated personae – paralan-
guage, dialogism and affiliation in stand-up comedy’. HUMOR: International Journal of Humor
Research and Logi, L. and M. Zappavigna (2019). ‘Dialogic resources in interactional humour’.
Journal of Pragmatics 153: 1–14.
Michele Zappavigna is Associate Professor in the School of Arts and Media at the University of
New South Wales. Her major research interest is the discourse of social media and ambient affili-
ation. Recent books include Searchable Talk: Hashtags and Social Media Metadiscourse
(Bloomsbury, 2018), Discourse of Twitter and Social Media (Bloomsbury, 2012), Researching the
Language of Social Media (Routledge, 2014, with Ruth Page, Johann Unger and David Barton)
and Discourse and Diversionary Justice: An Analysis of Ceremonial Redress in Youth Justice
Conferencing (Palgrave, 2018, with J.R. Martin).
24 new media & society 00(0)

Appendix 1. List of texts cited.

Text # Verbiage
7 its sad bc this is the standard for literally 90% of the media/pr/marketing
internships i see online
9 Youre such a little
10 Coffee before anything
11 Rainbow beach in Brisbane an expensive estate. Just opposite the Great Sandy
National Park, which house a steep trail towards a hidden aboriginal craving
site.
12 Love is love Proud of love.
13 Finally an excuse to post this photo I took that [name] refuses to post herself!
Happy 21st birthday @username
17 Happy birthday to my fave twins!!!! #thosewhogetfattogetherstaytogether
22 do we really have to do this
27 A belated Merry Christmas and a safe and happy holidays to y’all
31 Hyundai exec for Sale
35 Tragic
40 5 PLACES LEFT! There are only FIVE tickets remaining for BABSOC’s
Fungal Art Workshop! Secure your place now before tickets run out
43 Leighton Beach
44 no worries good luck !!!
45 Twinning it with this babe
52 Extremely chuffed that my first publication, an article for Discourse & Society, is
now online – a nice wrap on the first fortnight of my PhD
55 Thoroughly disappointed with the lack of milk
56 The good stuff
63 So i had salad
64 they’ve taken it off Netflix tho
65 You are welcome buy my tacos next week biatch
67 I can see how “interested” everyone is
72 Went to harry styles concert with 0 expectations but came back in awe???? What
a guy
73 chillssssss @username
77 Best day of my life so far ️
88 Take 3 for the sea (more is welcome too :);))
89 We are finalists in the category of Fitness Service in 2019 Melbourne North
Local Business Awards!
90 the cutest baby
99 Me & My worst frenemy.
111 Gonna miss this school Love every one of you boys, thanks for the
memories
115 been once, theyre pretty fun
147 Thanks so much for helping me out (and it was lovely to catch up last week!)
(Continued)
Logi and Zappavigna 25

Appendix 1. (Continued)
Text # Verbiage
169 Smaller than expected but cosy none the less (: Saw . I reckon it
would be a whole ‘nother scene at night. Wouldn’t mind coming back in spring
sometime as well with all the flowers in bloom~
176 Is anyone selling Taylor Swift tickets?
178 Embracing my skin more than ever because damn it’s bloody important;
sometimes you gotta let your skin breathe a little Also, my baby hair is getting
out of control, I can literally construct a fringe out of it
181 aww u r so wholesome
183 What a lovely thing to see! So happy you and kiddos are safe, it must be so
difficult for you to keep things together right now, but I just wanted to commend
you on your strength!
188 Do you want to today?
192 new nose piercing
198 If you want a break from studying and pet some animals there is a petting zoo
at the Physics Lawn (in front of Parker apartments) RIGHT NOW until 1pm !!!
It’s entirely free and there are trained handlers with lots of animals
200 so many freaks on here will literally jump at peoples throats to defend objectively
shitty people just bc they make mediocre samefaced art thats vaguely popular in
obscure internet circles it’s SO embarrassing. couldn’t be me