Professional Documents
Culture Documents
Introduction 1
1. Discourses, modes, media and meaning in an era of pandemic: A multimodal discourse
analysis approach 3
SABINE TAN AND MARISSA K. L. E
Introduction
Late in December 2019, the world first learnt about the outbreak of a cluster of viral pneumonia
cases in Wuhan Province, China. Soon after, on 9 January 2020, the World Health Organization
(WHO) reported that the Chinese authorities had identified a novel coronavirus as the cause of these
cases. The virus spread rapidly across the globe, and the out- break of the initially unnamed disease,
which eventually became known as COVID-19, was upgraded to a pandemic on 11 March 2020
(World Health Organization, 2020). No country was spared, and all aspects of everyday life have
been upended by the unfolding of the pandemic.
To stem the rise of infections, governments introduced lockdowns and border closures, and
international travel came to halt. People were com- pelled to stay at home and work and learn using
online video confer- encing platforms, wear masks and practice social distancing. To protect
themselves and others against the COVID-19 virus, the public needed to stay informed, following
advice given by official healthcare providers and government authorities amidst the proliferation of
‘fake’ news and misinformation circulating on social media channels. In short, the ways we think of
and experience the world, behave in it and express our varied realities have had to change rapidly in
tandem with the ever-evolving crisis.
This book examines various discourses, modes and media in circulation during this early period of
the COVID-19 pandemic, and how these have significantly impacted our daily lives in terms of the
various meanings they express that have come to form the (new) realities we experience and interact
with daily. Examples include how national and international news organisations communicate
important information about the virus and the crisis, the public’s reactions to such communications,
the resultant social media (counter-)discourses as reflected in social media posts and memes, as well
as the impact of social distancing measures on learning and education.
While the book offers a synoptic view of communication during the early phases of the COVID-19
pandemic, an examination of varied discourses, modes, media and meaning during the early days of
the pan- demic is necessary since it has led to new developments in the ways we came to experience
and adapt to a ‘new normal’. More than two years after the idea for this book was conceived, the
COVID-19 pandemic con- tinues to have a widespread impact on every sphere of our lives (e.g.
Nyland & Davies, 2022; Rogers, 2020; Timotijevic, 2020; Vyas, 2022).
Theoretical approach
To explore the varied meanings and realities engendered by COVID-19 discourses, this book adopts
a multimodal discourse analysis (MDA) approach. This approach, with its established theoretical and
analyt- ical frameworks – for example, Hodge’s (2016), Kress’ (2009) and van Leeuwen’s (2005)
social semiotics; Kress and van Leeuwen’s (2001) perspectives on the modes and media of
contemporary communication, and their seminal grammar of visual design (2006); Baldry and
Thibault’s (2006) and Norris’ (2011, 2014) conventions for transcription of multi- modal videos;
Machin’s (2007, 2013), Machin and Mayr’s (2012) and van Leeuwen’s (2008) multimodal
2
approaches and tools for critical dis- course analysis; and many others – is ideally positioned to make
use of the vast trove of data available as entire nations, societies and individuals across the globe
work to reframe, reorientate and recalibrate a world that is forever changed by a pandemic of
proportions that many have never ever seen before in their lifetimes.
The utility of the MDA approach stems from how it can be used to investigate different forms of
varied meanings that emerge from the many different discourses, modes and media interacting with
one another. The MDA approach is useful in that it also considers the interaction between text,
image, video and sound typical of the digital era, and is thus an apt tool for examining how
discourses about the pandemic were represented across different spheres of professional and private
lives.
While discourses on multimodality have developed distinct and idiosyncratic – and sometimes
contested – interpretations of the terms ‘discourses’, ‘modes’, ‘media’ and ‘semiotic resources’ (e.g.
Bateman et al., 2017; Jewitt, 2016; Kress, 2009; Kress & van Leeuwen, 2001; van Leeuwen, 2005),
in this book these terms are approached from a more general perspective with a view to how these
are employed in COVID-19 discourses as resources for communication and meaning making.
speech acts pertaining to the COVID-19 pandemic were deployed multimodally in the cartoons. The
findings reveal that these extend beyond acts of judging and include many positive reactions and
behaviours such as thanking healthcare workers for their efforts and sacrifices in fighting the
coronavirus.
Chapter 3, COVID-19 communication as ‘graphic medicine’: A multi- modal social semiotic
approach, by Marissa K. L. E and Sabine Tan, explores the interactive meaning-making potential of
comics as graphic medicine (e.g. de Rothewelle, 2019; Green & Myers, 2010) by analysing a
selection of short comics from “The COVID-19 Chronicles” published by the National University of
Singapore (NUS). Adopting a multimodal social semiotic approach based on metafunctionally
organised frameworks as developed by Kress and van Leeuwen (2006, 2021) and van Leeuwen
(2005) for the analysis of images and other multimodal resources, the study investigates what makes
comics containing public health informa- tion about COVID-19 a potentially effective medium of
communication. The findings show how the strategic co-deployment of text and image in these
comics, together with the utilisation of the narrative form, helps to
(a) construct visual stories that simplify and gradually develop complex concepts for the general
public’s consumption, (b) engage audiences using affective appeal via humour and positive
appraisal, and (c) represent par- ticular actions taken by the Singapore government positively,
particularly with a view to the roles played by healthcare workers in managing a healthcare crisis
caused by the outbreak of COVID-19 in migrant worker dormitories.
(a) identify the most common phrases used by UK public health agencies to provide public health
information about COVID-19 on Twitter; (b) iden- tify the common images used with these phrases;
and (c) understand how both phrases and images function in these public health discourses. The
analysis showed that the most frequently used n-grams in these tweets often served to give advice or
instructions while presenting this more neutrally as information. The images that were combined
with these fre- quent n-grams tended to be photographs representing members of the public and
authority figures, procedural infographics, and posters with cartoon images. The authors observe that
the text in the tweets some- times appeared to be misaligned with the intended functions of the visual
representations (e.g. see also Chapter 11, this volume), thus affecting the overall effectiveness of the
public health communications.
rhetorical work, but as enactments that help to perform socially conscious brand personalities, with
brands not only presenting themselves as responsible corporate actors, but also as social citizens
attending to the distress of fellow citizens.
Chapter 8, Defamiliarise to engage the public: A multimodal study of a science video about COVID-
19 on Chinese social media, by Yiqiong Zhang, Rongle Tan, Marissa K. L. E and Sabine Tan,
focuses on the ways the rhetorical strategy of defamiliarisation (Shklovsky, 1917/2017) was
employed as an engagement strategy in a popular science video about the COVID-19 pandemic on
Chinese social media. Drawing on analyt- ical perspectives from multimodal discourse analysis and
social semiotics (e.g., Kress & van Leeuwen, 2001, 2006), the authors examine how the strategy of
defamiliarisation is realised semiotically in the video to frame and position subjects and events in
particular ways. By examining the multimodal construction of meaning at different levels of
discourse in the video, the analysis revealed that defamiliarisation operated on all three levels of
meaning – textual, interpersonal and ideational, as realised, for example, through juxtapositions of
meaning in the video’s structural organisation, different types of visual display in terms of their
modality value, the use of voice of authority and nostalgia, and elements of unex- pectedness in the
representations of social actors. The approach allowed the authors to demonstrate how the strategy of
defamiliarisation can potentially facilitate audience attention and affective engagement, while at the
same time enabling the communication of scientifically accurate and timely information. The chapter
highlights how a multimodal con- ceptualisation of defamiliarisation can offer a new perspective for
analysts and practitioners in science communication who are interested in how to engage the public
in novel and unfamiliar ways amidst the ‘infodemic’ of science news about the coronavirus and the
pandemic.
In Chapter 9, Beyond reporting: The communicative functions of social media news during the
COVID-19 pandemic, Yuanzheng Wu and Dezheng (William) Feng present findings from an
analysis of 232 news video clips posted by CCTV (China Central Television) News on TikTok to
highlight the multiple functions social media news perform in engaging the public with news about
the COVID-19 pandemic in China. Drawing on Matthiessen’s (2009) register typology of social
semiotic activities and Martin and White’s (2005) system of Appraisal, the authors demonstrate that,
apart from reporting the news, social media news performs a wide range of communicative
functions, such as sharing positive attitudes and emotions about the reported content, expounding
pandemic-related knowledge and recommending appropriate protective measures. The findings show
that, in their corpus, sharing was the most prominent activity, which serves to distinguish social
media news from traditional news. In terms of the realisation of social semiotic activities, the
findings also showed that the news posts were characterised by a prevalence of personalised and
emotional content in both language and images. As such, the study presents new insights into the
multiple activities and discursive features that served the social, political and educational functions in
man- aging the COVID-19 pandemic in China as shaped by the affordances of the social media
platform TikTok.
Chapter 10, Exploring strategies of multimodal crisis and risk commu- nication in the business and
economic discourses of global pandemic news, by Carmen Daniela Maier and Silvia Ravazzani,
endeavours to broaden the range of methodological approaches to multimodal communication by
proposing an interdisciplinary approach for investigating multimodal crisis and risk communication
in business and economic discourses that have appeared in the news media during the first months of
the COVID- 19 pandemic. Combining the social semiotic approach to multimodal discourse (van
Leeuwen, 2008, 2018a, 2018b) with approaches to crisis and risk communication (e.g. Coombs,
2012; Heath & Palenchar, 2016; Raupp, 2018), the authors develop an analytical model for revealing
the ways in which multimodal business and economic discourses in global news media function to
6
legitimate and evaluate various versions of the unpredictable reality across blurred national borders.
They then apply this model to a selection of global news videos released during spring 2020 by three
international news channels, namely, CNN, The Economist and Financial Times, to examine and
reveal the fundamental role played by social actors as risk informers (e.g. journalists), risk bearers
(e.g. consumers), risk researchers (e.g. economists) and risk regulators (e.g. politicians) as they enter
the arena of global news with multiple discursive agendas in the context of the COVID-19 pandemic.
By employing this taxonomy of risk-related roles, the authors demonstrate the approach’s utility in
enabling them to explain how the news media’s discursive selection of specific social actors with
specific expertise and experiences contributes to making sense of the pandemic’s risks and their
impacts from multiple perspectives. The findings also affirm the value of inte- grating insights from
risk communication research with a multimodal discourse analysis approach.
In summary, the themes and topics around which this book is organised encapsulate the pandemic’s
effects across different dimensions of society, from the political, to the economic and the social. As
can be seen from these chapters, the pandemic has influenced communication patterns via the use of
semiotic modes and resources, and media technologies, conse- quently impacting communicative
functions and strategies, and leading to the construction of wider communicative meanings and
purposes. Such an influence, we argue, will have long-lasting effects as a result of the continuation
of the pandemic, with the disease likely becoming endemic. The phenomena examined in these
chapters, such as online learning, citizen sociolinguistics and de-legitimisation through the use of
humour, will thus continue to remain relevant in the foreseeable future. With the possibility of future
pandemics on the horizon, presenting this collection of COVID-19 discourses for examination to our
readers provides us with an opportunity to consider and prepare for issues involving multimodal
communication that will likely arise in similar situations to come.
PART I
Use of semiotic modes/resources in COVID-19 discourses 15
2. ‘Stay at home’: Speech acts in Arab political cartoons
behaviours useful in the maintenance of public health via affective engagement strategies such as
humour. Furthermore, the analyses also showed how comics played a part in helping to manage a
healthcare crisis caused by the outbreak of COVID-19 in migrant worker dormitories by focusing on
the roles played by healthcare workers and migrant workers in making sure the outbreak did not
escalate further.
With the end of the pandemic yet in sight and the possibility of further viral variants emerging, there
remains a need for effective public com- munication of scientific and health information about
COVID-19. This is especially true with pandemic fatigue setting in, and as a result, indi- viduals
potentially being desensitised to public health communication about the COVID-19 virus. Perhaps
comics as graphic medicine, with their potential to provide viewers with culturally and contextually
rele- vant and accessible information that utilises affective engagement, can help address such a need
by breaking through the tedium, worry and fear of a continued pandemic situation.
PART II
Use of media/media technologies in COVID-19 discourses 67
4. Design considerations for digital learning during the
COVID-19 tweets 89
DAVID OAKEY, CHRISTIAN JONES AND KAY L. O’HALLORAN
The research reported in this chapter has revealed that in UK public health agency tweets, linguistic and
visual elements are, predictably, interconnected through the common themes of preventing the spread of
COVID-19 through safety measures (i.e. working from home, getting tested, keeping socially distanced,
and washing hands). However, we have observed that the tweet texts and embedded linguistic text are not
always perfectly aligned with the images, especially in cases where infor- mation is provided in the form
of factual statements that are only vaguely related to the image. In this case, the most frequent n-grams
form core aspects of the messages, but these often function indirectly, and their illocutionary force can be
interpreted in various ways, unlike the pro- cedure graphics where the conditions and logical implications
are clearly articulated. However, procedure infographics are complex multimodal texts and need to be
directly linked to the accompanying tweet text, to be more effective. Additionally, there is a clear risk of
9
redundancy and infor- mation overload if the tweet text merely repeats what is depicted graph- ically, and
a risk of misinterpretation if the text is repeated with minor changes within the graphics. While public
health tweets of this nature need to communicate their message quickly and with minimal ambiguity,
overall, the examples identified in this study suggest that the manner in which the most common n-grams
and accompanying images are used reveals that there is a great deal of ambiguity present.
This stage of the project has described the language and images used in public health messaging and the
nature of the text/image relations. As seen in this study, the text and images can function to co-
contextualise each other, in this case using the tweet text and images such as photographs, procedural
infographics, and posters. Key to such an approach is to ana- lyse the message through a multimodal lens,
analysing common forms and functions of language and how these interact with images to carry
ideational, interpersonal and textual messages. Although our focus has been limited and reasonably
narrow here, the language/image analysis methodology developed in this project can be applied to all
sources of information about COVID-19 (mainstream news, social media, govern- ment reports and so
forth). Insights into how phraseology and imagery in public health information are understood and acted
upon will lead to increased effectiveness of the linguistic and visual choices made in ensuing messages in
future pandemics.
PART III
Communicative functions/strategies of COVID-19
discourses 115
6. Australian universities engaging international students during the COVID-19 pandemic:
A study of
multimodal public communications with students 117
ZUOCHENG ZHANG, TONI DOBINSON AND WEI WANG
We began the study by asking how Australian universities engaged their international students amid the
COVID-19 pandemic and how a situated discursive practice perspective could shed more light on
university and student engagement. Our case studies of the engagement practices of the University of
New England, Curtin University, and the University of Sydney, through their public communications,
indicate that all three focal universities displayed high levels of pastoral care for their inter- national
students. This is attested in the large number of Informing and Supporting communicative acts observed
in the communications of the universities and the larger number of such communicative acts at critical
stages of the pandemic. The focus of the universities, through the com- municative efforts of key
academics such as the VCs, indicates a univer- sity stance which values connection, greater intimacy and
caring, which is in line with the literature which calls for a re-humanising of university academics and
their intersubjective experiences with students (Gilmore & Warren, 2007).
However, other roles included in Cognitive engagement, such as promoting thinking, critiquing and
reflecting, were downplayed by all three universities. This is evident in the minimal Cognitive
engagement instances identified in the linguistic and visual texts. While a calibrated degree of action is to
be expected in these extraordinary times of an unpre- cedented, urgent, high-risk situation, where rules
need to be conveyed and obeyed for the safety of all, and where student mental health is at stake, this
stance still challenges the accepted role of universities as encouragers and promoters of critical thinking,
in particular, during a crisis situation. This critique resonates with Zepke’s (2018) call to take student
engagement beyond prevailing neoliberal practices of focusing on engagement for practical ends (e.g.
10
8. Defamiliarise to engage the public: A multimodal study of a science video about COVID-
19 on
Chinese social media 160
YIQIONG ZHANG, RONGLE TAN, MARISSA K. L. E AND SABINE TAN
This study has adopted the concept of defamiliarisation, interpreted from a social semiotic perspective, to
analyse the multimodal meaning construction in a science video about the COVID-19 pandemic on
Chinese social media. By showing the multimodal realisations of the rhetorical strategy of
defamiliarisation at different levels of discourse in the video, we have revealed how this strategy was
used effectively to represent subjects and events in “strange” and unexpected ways, requiring viewers to
re-examine their perceptions of what would ordin- arily seem conventional and familiar. The application
of the concep- tualisation of defamiliarisation in studying science communication discourse brings in a
new perspective for analysts and practitioners in science communication concerning how to engage the
public in the age of an “infodemic” perpetuating by unverified and unsubstantiated infor- mation about
the COVID-19 pandemic. The role of visual resources in achieving the effect of defamiliarisation
suggests that non-verbal resources are more likely to be used for emerging discourse practices, though
meanings are always constructed multimodally. While we are fully aware of the limitations of the present
study with only one video being analysed, we hope that the particular case will help to stimulate
reflections for further exploration in the increasing complex environ- ment of science communication.
Such reflections are more pressing than ever in this “infodemic” era.
12
10. Exploring strategies of multimodal crisis and risk communication in the business and
economic
discourses of global pandemic news 201
CARMEN DANIELA MAIER AND SILVIA RAVAZZANI
Our analysis offers several points of reflection on the strategies of multi- modal crisis and risk
communication in the business and economic discourses deployed in global news videos during the first
months of the current global pandemic crisis. When various risk-related social actors are recontextualised
from reality into the business and economic discourses in the global news, the multimodal
recontextualisation takes various forms. The risk gen- erator, that is the virus, is obviously at the centre of
such discourses and of the sensemaking processes that have been discursively employed in all videos.
The multimodal recontextualisations of this risk generator are meant, in particular, to highlight the
dramatic consequences for risk bearers and the implications for risk researchers and risk regulators. The
verbal and/or visual identification, nominalisation and functionalisation of risk informers are meant to
legitimise the validity of their statements and the additional visual information that is displayed on-screen
related to other social actors and/or actions. The various multimodal recontextualisations of risk bearers
as identifiable individuals, unidentifi- able individuals or associated groups are meant to witness their
existence and role in the pandemic crisis both in specific situations and globally. The recontextualisations
of both risk researchers and risk regulators are meant to legitimise multimodally the sensemaking
processes that are employed when crisis-related social actions are in focus. From our interdisciplinary
perspective, these findings emphasise the soundness of integrating in multimodal discourse analysis the
13
social dramaturgical approach belonging to risk communication research to distinguish and make sense
of the variety of agents and risk-related roles allocated to them in news media discourses (Palenchar et
al., 2014; Palmlund, 2009). The retrospective, synchronous and prospective sensemaking processes are
discursively materialised especially through multimodal theoretical or effect-oriented rationalisations that
evaluatively clarify the causes, development, probable consequences and possible future solutions of the
business and economic havoc created by the global pandemic crisis. These findings underline the validity
of adopting, in an interdisciplinary effort, also the sensemaking approach belonging to crisis
communication research to label, stabilise (Weick et al., 2005) and interpret complex phenomena such as
crises as multimodally represented in news media discourses (Höllerer et al., 2018; Maier & Ravazzani,
in press). All in all, these multimodal recontextualisation strategies shape communicatively the viewers’
understanding of the global pandemic crisis by enabling or constraining certain sensemaking processes
and by including specific risk- related social actors. Such multimodal recontextualisation strategies in
global news media discourses have not only communicative functions but also constitutive ones, as
certain unsettling versions of reality emerging from the screen are supposed to be accepted by bewildered
and worried audiences due to the discursive force of these videos.
The empirical material and the systematic analysis explained in this chapter show that a call for drawing
upon a broader set of theoretical perspectives to be included in and combined with multimodality
research could be appropriate. In this chapter, combining multimodal discourse research with theoretical
perspectives from other fields such as crisis communication and risk communication has given us the
possibility to reveal how social actors and social actions are recontextualised by global news media to
make sense of an unprecedented crisis with still incompre- hensible risks. Similar interdisciplinary
approaches could be influential also in future research, for example by explaining and combatting the
dangerous multimodal infodemic (Gao & Basu, 2020; WHO, 2020a) of the present pandemic era that is
spreading across all media; as well as by delving into the multiplicity of voices that communicate to,
with, past, against or about each other (Frandsen & Johansen, 2017) in virtual dis- cussion arenas such as
social media. To conclude, based on the present research endeavour, we consider that such
interdisciplinary approaches can complement, reinforce and refine the existing multimodality research
agenda.
PART IV
Wider communicative meanings/purposes of COVID-19 discourses 225
11. “Stay Alert, Control the Virus, Make Memes”:
may reveal significant insights about human behaviour during the pandemic – for example, if a certain
meme prompted a behaviour change that directly impacted the number of cases in a region.
While films, television, books and other media are certain to emerge as a creative record of life in the
pandemic, they may not fully encompass all of the public discourses that emerged during this time. The
public view of “Stay Alert” is an important part of the COVID-19 era, and this view is captured clearly in
social media discourse, including the memes in this chapter and others. As the analysis has shown,
memes can subvert and delegitimise the government’s official narrative. Uncovering the meaning of
these memes is necessary if we are to fully understand the public’s experience of the COVID-19
pandemic.
as well as other voices. In this, the affordances of social media platforms offer some advantages.
They allow for open conversations drawing in different voices through the sharing of links and
multimodal forms of expression, thus enabling co-construction of knowledge beyond writing,
through multimodal legit- imation also by showing, through “demonstration” (for example, the video
that Anzir shared as a comment, to make his point in the discus- sion). While we all observe, discuss
and comment on semiotic practices all the time, offline as well as online, social media enable reach
beyond “the pub chat”. Firstly, because they make possible the transnational coming together of
diverse voices in “affinity spaces” (Gee, 2005). This affords the consequent potential sharing and
spreading of semi- otic practices beyond the limitations of physical space and movement. Besides the
case of the backpack hug, which “travelled” from Italy to Canada, many discussion threads in the
PanMeMic Facebook group had participants sharing semiotic practices taking place in their coun-
tries, with other members commenting on how these compared with practices in their own areas. It
also enables views beyond those in our immediate personal networks to interact. In the discussion on
masks and sign language, for example, Emma’s intervention as a hearing-impaired person offered a
key perspective on the issue for the other – all hearing – participants to consider, potentially
enriching their exposure to diverse experiences beyond not only national but also other sociocultural
group boundaries. Secondly, social media make it possible to track and record as well as offer open
access to such interactions, to an extent that would be difficult to achieve for interactions occurring
offline.
The case of PanMeMic is also different from more traditional (multi- modal or not) discourse
analysis of online discussion data. As researchers, we not only offered a space for but participated
ourselves in informal discussions with other members of PanMeMic (see Elisabetta’s comment in the
second example analysed). This has created opportunities for all of us, both multimodality scholars
and those who have a personal interest in discussing semiotic and interactional practices, to engage
in a col- lective inquiry and invite others to join the dialogue. We further invited members of the
group to verify our interpretations of the discussion and to check, further discuss and contribute to
the analysis presented in this chapter. We joined in the conversations with others, generating
relational ways of inquiry and mutual learning that are very different from both the methods and the
outcomes of any textual analysis or covert observation methodologies.
Responding to the theme and concerns of this volume, we hope our contribution has shown that
discourse analysis can be used not only to identify how discourse shapes reality ideologically, as
critical discourse analysts do, but also, as in our case, to find insights into semiotic practices that are
being co-constructed, shared and legitimated while being talked (or written) about. Even more, our
findings show everyday people’s contributions as discourse analysts, as the exchanges examined
reveal analytical insights and critical approaches to discourse that people show when participating in
discussions. Together with its usefulness, our ana- lysis shows also the limitations of discourse
analysis, and how involving the text producers themselves offers further insights as, in the backpack
hug example, confirmation that a semiotic practice being talked about has actually been used by one
and has instead been replaced by its first proponent. In sum, we believe we have offered a glimpse
into the benefits of adopting an ethnographic, engaged and involved approach to partici- pating in
discourse and its analysis.