Journal of Second Language Writing 57 (2022) 100869

Contents lists available at ScienceDirect

Journal of Second Language Writing

journal homepage: www.elsevier.com/locate/jslw

Developing a genre-based model for assessing digital multimodal

composing in second language writing: Integrating theory
with practice
Lianjiang Jiang a, Shulin Yu b, *, Icy Lee c
a
Faculty of Education, The University of Hong Kong, Hong Kong, China
b
Faculty of Education, University of Macau, Macau, China
c
Faculty of Education, The Chinese University of Hong Kong, Hong Kong, China

A R T I C L E I N F O A B S T R A C T

Keywords: While previous research on digital multimodal composing (DMC) has examined the efficacy of
Digital multimodal composing either an element-based rubric or a process-based model that assesses students’ DMC across
Writing assessment stages, the important notion of genre and its value for DMC assessment remains underexplored
Genre-based model
and undertheorized. Given that as a new literacy practice in L2 writing and related fields, DMC
Writing instruction
covers a wide range of genres, a genre-based model that incorporates the composing elements and
process for assessing DMC is warranted. Driven by theories of genre and multimodality, the study
first proposed a multilayered framework that entails DMC structures, functions, modal features
and selections. Then the theory-driven framework was tested and modified through collaborative
action research with five teachers of a university English for general and academic purposes
course in China. The study then drew on student-authored multimodal compositions, interviews
and observations with the teachers in order to explore how the teachers assessed DMC, as well as
the challenges they encountered. Based on the findings, a refined genre-based model that guides
teachers to evaluate DMC as purpose-directed social actions to be constructed with apt multi­
modal choices within and across four major layers (i.e., base units, layout, navigation, and
rhetoric) was developed, with implications discussed.

1. Introduction

Along with the advancement of social media and digital tools, the importance of digital multimodal composing (DMC) as a new
form of writing genre and composing practice for second language (L2) writing in the digital age is well documented (Belcher, 2017;
Elola & Oskoz, 2017; Jiang, 2021; Yi, Shin, & Cimasko, 2020). By definition, DMC refers to a textual practice that involves the use of
digital tools to interweave words with multiple semiotic modes such as image and soundtrack (Hafner & Miller, 2019; Jiang, 2017).
Typical examples of DMC across L2 writing contexts include composing of web pages, podcasts, various forms of videos, infographics,
or posters (Hafner & Miller, 2019; Yi et al., 2020). In recent years, considerable work (e.g., Hafner & Miller, 2011; Jiang, 2018; Li,
2020; Jiang & Luk, 2016; Smith & Shen, 2017) has gone into the illustration of how DMC may be conducive to various aspects of
writing, such as writing motivation and writer identity. There has been an ongoing call for L2 writing practitioners and educators to

* Corresponding author at: Faculty of Education, University of Macau, Room 3007, E33, Macau SAR, China.
E-mail address: shulinyu@um.edu.mo (S. Yu).

https://doi.org/10.1016/j.jslw.2022.100869
Received 15 May 2021; Received in revised form 27 December 2021; Accepted 4 January 2022
Available online 21 January 2022
1060-3743/© 2022 Elsevier Inc. All rights reserved.
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

expand the traditional L2 writing curriculum and include DMC as one instructional design so as to prepare L2 learners for
technology-mediated writing practice in the new century (e.g., Kim & Belcher, 2020; Yi et al., 2020).
Despite the illuminating findings and calls, one unsettled challenge for L2 writing teachers and researchers is how to assess DMC,
which diverges from traditional writing in its involvement of images, videos, words, speech, and other forms of representation modes
(Hafner & Ho, 2020). To explore the issue, previous research has mainly examined the efficacy of either an element-based rubric that
evaluates students’ deployment of linguistic, gestural, spatial, visual, and auditory elements (Hung, Chiu, & Yeh, 2012) or a
process-oriented model that assesses students’ DMC across four stages, i.e., pre-design, design, sharing, and reflecting (Hafner & Ho,
2020). Yet the important notion of genre and its value for DMC assessment remains undertheorized and underexplored. Given that as a
new literacy practice in L2 writing and related fields, DMC covers a wide range of genres, including but are not limited to docu­
mentaries, posters, digital stories, and multimodal presentations (Zhang, Akoto, & Li, 2021), a genre-based model that incorporates the
composing modes and process for assessing DMC is warranted.
This study intends to contribute to L2 writing assessment in the digital age by developing a genre-based model for DMC assessment.
Driven by theories of genre and multimodality, the study first conceptualized a genre-based model that entails DMC structures,
rhetorical functions, modal features and selections across layers of base units, layout, navigation and rhetoric. Then the theory-driven
model was tested and modified through collaborative action research with a cohort of teachers of a university English for general and
academic purpose course in China. The study aligns with the recent call for more research on assessment practice for DMC in the L2
writing context (Yi, King, & Safriani, 2017). It is also a timely response to the scholarly observation that a genre-based model would be
a good place to start in developing DMC assessment criteria that account for both the multimodal affordances of digital media and the
generic applicability to a range of different genres of multimodal compositions (Hafner & Ho, 2020).

2. Assessing DMC: existing approaches and challenges

Previous attempts to develop assessment strategies for students’ design of multimodal compositions can be summarized as either
element-based or process-oriented. The element-based approach is mostly informed by New London Group’s (1996) theory of mul­
tiliteracies and it takes the deployment of semiotic resources in linguistic, visual, gestural, auditory, and spatial modes as the foci of
assessment. For instance, situated within a Taiwanese university writing context, Hung et al. (2012) designed a theory-driven rubric
that evaluates whether students’ use of five semiotic modes (e.g., linguistic, gestural, auditory, visual, spatial) enables or limits
meaning expressions and multimodal cohesion in their production of multimodal texts. Through an 18-week course evaluation, Hung
et al. (2012) observed that their element-based rubrics can be used as a formative assessment tool, with positive effect upon L2
learners’ multimodal composing. Nevertheless, such rubric was developed based on one genre of DMC (i.e., presentation slide) and it
does not emphasize language proficiency and assessment (Yi et al., 2017); thereby such rubric may risk being undervalued by L2
writing professionals in practice.
The process-oriented approach was synthesized by Hafner and Ho (2020) based on their exploration of teachers’ assessment
practice for DMC in a university-based English for science course in Hong Kong, where students were guided to produce videos of
scientific documentaries. In addition to students’ orchestration of multimodal elements, this practice-based model showcases how
DMC assessment can be planned across four stages of the composing activity (i.e., pre-design, design, sharing, reflecting) and specifies
when formative and summative strategies can be drawn upon throughout the process. The pre-design stage forms the basis of
assessment and it involves composing plans and notes. The design stage focuses on students’ generation of scripts, storyboards and
other media. In the sharing stage, student-generated multimodal composition is attended to as product with self-, peer-, and
teacher-led assessment activities. The focus for the reflecting stage is on students’ awareness and transfer to other writing contexts
(Hafner & Ho, 2020). While this process-based model is grounded in classroom practice and insightfully illustrates what and when to
assess, for better implementation of such a model, there is a clear need for teachers and students to develop the ability to “explain how
a particular selection of multimodal semiotic resources contributes to the overall meaning required for a particular genre and medium”
(Hafner & Ho, 2020, p. 12). There is also a need to cater for the competence needs of L2 writing students to combine language with
other modes through multimodal orchestration by selecting apt resources appropriate to genre and purpose (Yi et al., 2017). With such
awareness, Hafner and Ho (2020) called for more research on a genre-based model for the assessment of DMC.
Nevertheless, the notion of genre is mentioned in a rather brief manner in previous works on assessing DMC. For instance, in their
work on teachers’ assessing student-authored multimodal compositions, Hafner and Ho (2020) noted that genre was one of the seven
criteria used by the teachers, with others being creativity/originality, organization, language, delivery, modal interaction, and variety.
It is suggested that the notion of genre in print-based writing can be expanded to account for genre in digital writing (Askehave &
Nielsen, 2005; Giltrow & Stein, 2009; Hafner & Ho, 2020). For print-based writing, a genre-based model normally focuses teach­
ers/evaluators on issues of whether organizational, grammatical and lexical choices would meet the expectations of target audience
and achieve the communicative purposes within certain discourses or social communities (Hyland, 2007; Swales, 1990). For digital
and multimodal writing, Hafner and Ho (2020) indicated that the model can be expanded to consider whether and how the affordances
of digital media and modes have been utilized with apt multimodal choices that fulfill target audiences’ expectations and any other
social and disciplinary functions. Similar reference to the notion of genre is also evident in Burnett, Frazee, Hanggi, and Madden’s
(2014) rubric to evaluate multimodal processes and artifacts. The rubric comprised five dimensions (i.e., rhetorical awareness, stance
and support, organization, convention, design for medium) and genre was taken as only one subfactor within the category of
convention, without a clear definition of what genre is in their rubric. It also remains unclear how a genre-based model can be
developed and applied to DMC assessment and what challenges, if any, may be experienced by teachers in their classroom-based
practice.

2
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Previous research suggests that assessing DMC, or more broadly, multimodal literacy practice, is a challenging enterprise (Yi et al.,
2017). It can be imagined that evaluating DMC with a genre-based model of assessment, which is underexplored, may present even
more challenges. General challenges in multimodal assessment include a deeply entrenched belief among L2 practitioners about the
dissonance between traditional assessment of writing that privileges the print-based linguistic mode and the assessment of DMC that
takes into account both linguistic and non-linguistic modes (Jiang, Yu, & Zhao, 2019; Mills & Exley, 2014). There are also concerns
over the potential tensions between the individually-based standardized testing practice and the usually collaborative nature of DMC
(Mills & Exley, 2014; Yi et al., 2017). The third challenge is about the indeterminacy regarding what constitutes legitimate/valued
knowledge and indicators of learning during digital/multimodal learning activities (Wyatt-Smith & Kimber, 2009). Given the pre­
dominant focus on language in traditional writing assessment in pen-and-paper environments, L2 teachers usually experience ideo­
logical struggles over the legitimate evidence of learning in assessing DMC in L2 writing contexts (Jiang & Ren, 2020). More
fundamentally, teachers are reported to lack confidence in teaching and assessing DMC, which is exacerbated by the lack of relevant
teacher training (Jiang et al., 2019; Li, 2020; Yi & Choi, 2015). In light of these challenges, assessing DMC with a genre-based model
requires the development of flexible and innovative methods and perspectives. While some scholars (e.g., Hafner & Ho, 2020) suggest
that the notion of genre in print-based writing can be expanded to digital and multimodal writing, we believe that extending the notion
of genre directly from linguistics to multimodality may be problematic because traditionally, the structure of genre is often taken as
linear with predetermined reading paths (e.g., from left to right). In multimodal environments, however, there is evidence suggesting
that visual perception is normally task-driven and unlikely to follow a rigid predetermined pattern (Hiippala, 2012; van Leeuwen,
2005). Thus, novel approaches are warranted if we want to expand genre-based assessment and analysis from traditional writing to
DMC. To this end, the present study with two stages was conducted. For stage one, we developed a multilayered framework that entails
DMC structures, functions, modal features and selections. For stage two, the theory-driven framework was tested through collaborative
action research with five teachers of a university English for general and academic purposes course in China, and the study then drew
on student-authored digital multimodal compositions, interviews and observations with the teachers in order to explore how the
teachers assessed DMC, as well as the challenges they encountered.

3. Stage one: towards a multi-layered multimodal genre assessment framework

In this study, we draw on the Genre and Multimodality model (i.e., GeM, Bateman, 2008) to conceptualize a genre-based model for
DMC assessment. The GeM model, originally developed by the Genre and Multimodality research project at the University of Stirling and
Bremen University, employs the notion of genre to evaluate and analyze variations in multimodal composition structures (Hiippala,
2017). Following the traditional understanding of genre as imposing context-specific constraints on the selections made within lan­
guage and discourse (Hyland, 2007; Waller, 1987), the GeM model proposes the notion of multimodal genre, which might similarly
exert constraints on selections within layout structures and on their typographical and spatial realization (Bateman, 2014). From
inception, the GeM model has been intended to “function as a tool for isolating significant patterns against the mass of detail that
multimodal documents naturally present” (Bateman, 2014, p. 33). It provides a systematic empirical exploration of multimodality for
formulating theories about the principles of organization and configurations in multimodal compositions (Hiippala, 2017). It also
echoes Prior’s (2009) point that all genres are multimodal.
While the GeM model was originally proposed for the purpose of annotating multimodal documents (Bateman, 2008; Hiippala,
2017), it suits our purpose of assessing DMC for three main reasons. First, it takes genre as social semiotic, a central concept of which is
mode. According to Bateman (2014), the first step in multimodal analysis involves an effort to identify the semiotic modes, which
entail an underlying material substrate that carries a number of semiotic resources, the meaning of which should be interpreted in
context (Hiippala, 2017). Considering the materiality of modes, students’ DMC and multimodal choices can be constrained by a
combination of physical, production and consumption constraints (Bateman, 2008). Physical constraints refer to constraints arising
out of the materiality of modes (e.g., paper or screen size). Production constraints refer to constraints arising out of the production
technology and micro and macro economies of time or materials (e.g., availability of images, deadline). Consumption constraints arise
from generic/social expectations (e.g., that the artifacts must be easy to understand) and assumptions of expertise of read­
ers/consumers. These constraints are relevant for DMC assessment because they enable a more realistic evaluation of the motivations
and reasons for DMC, which can be described as a compromise between constraints (Bateman, 2008).
Second, the GeM model takes genre as social action (Miller, 1984; Swales, 1990). Following Swales’ (1990) classical definition,
genre is taken as “a class of communicative events that share a recognizable communicative purpose, that exhibit a schematic structure
supporting the achievement of that purpose, and which show similarities in form, style, content, structure and intended audience” (p.
58). In terms of schematic structure, genre can be divided into moves and stages and is closely related to rhetorical strategies and
organizations (Bateman, 2008; Swales, 1990). As Kress and Van Leeuwen (2006) emphasized in their accounts of visual grammar,
since both linguistic and multimodal artifacts are social practice and indicative of social relations and rhetorical organizations, the
notion of genre can be a useful category to apply regardless of mode (Bateman, 2008).
Third, the GeM model is relevant to our study because it takes genre as a multilayered and multi-stratal phenomenon (Hiippala,
2017). As described by Bateman (2008), the primary layers of the GeM framework include structures of content (i.e., the content
related structure of information to be communicated), genre (i.e., the stages/moves of a given genre), linguistic (i.e., linguistic details
of any verbal elements), rhetorical (i.e., how content is divided into main materials and supporting materials), layout (i.e., the
appearance and position of communicative elements and their hierarchical relationships), and navigation (i.e., elements that serve to
direct or assist intended ways of consumption). These six structures offer a framework for the “layered decomposition” (Bateman,
2008, p. 19) of any multimodal documents and artefacts (e.g., film, video). Through such decomposition, it is possible to evaluate how

3
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

the particular selections and choices across different strata are socially motivated and serve to achieve the designated communicative
goals or social purposes (Bateman, 2008; Kress, 2010).
Based on the above considerations and aligning with the primary layers in the GeM model (Bateman, 2008), we developed a
multilayered framework for DMC assessment (see Table 1). This framework contains four sublayers: base layer, layout layer, navi­
gation layer, and rhetorical layer. The base layer includes a list of base units such as sentences, captions, images, audio, video clips, and
color. At this layer students need to attend to the material aspects of each mode and decide what modes to select and use in their DMC.
The layout layer refers to the hierarchical structure, if any, of the identified base units and their positioning (i.e., placement) in the
layout. At this level, students need to decide whether and how the base units (e.g., linguistic or visual modes) may concur, comple­
ment, or conflict with each other in terms of coherent meaning expressions (Hafner & Ho, 2020; Unsworth, 2007). The navigational
layer is related to the navigational structures, which are provided to target readers/consumers for the intended use or consumption of
digital multimodal compositions. Depending on the overall communicative purposes (e.g., to persuade, to inform), students at this
layer need to be concerned about the overall stages or phases to be included for a given genre (e.g., narrative, argumentation). In
addition to determining the rhetorical relations and layout positions of various base units, students need to decide closely how the
delivery of content proceeds through some recognizable stages or moves of activity (Bateman, 2008).
The rhetorical layer is directly related to meaning and it assumes that combinations of base units are normally governed by
meaningful relationships and communicate certain functions (Bateman, 2008). For instance, one rhetorical relation between two base
units, e.g., image and word, can be elaboration, with the image elaborating the word meaning. Such rhetorical relations and functions
can be described through Rhetorical Structure Theory (RST), which defines a set of around 25 rhetorical relations such as background,
circumstance, elaboration, evidence and clause (see Bateman, 2008 for details) that are commonly observed in natural texts (Mann &
Thompson, 1988). These relations can be divided roughly into asymmetric and symmetric relations. In asymmetric relations, one
rhetorical unit can be singled out as the most important, or nuclear, component, with the other subparts called satellites. In symmetric
relations, all the related units are treated as equally important. Examples of symmetric relations include contrast (i.e., a relation
between two equally important elements) and sequence (i.e., a relation that brings together a number of elements into a temporal or
other sequence) (Bateman, 2008; Hiippala, 2017).
Overall, we align with Bateman and colleagues and highlight the utility of treating DMC as rhetorically organized structures that
combine modal contributions in the achievement of communicative goals (Bateman, 2008). On the basis of Bateman’s (2008) GeM,
which was originally designed for page-based documents, our multi-layered preliminary framework expands multimodal genre from
writing in print-based multiple modes to digital multimodal composition and media production. With a focus on the digital medium,
we believe that digital genres are fundamentally different from print-based genres (Giltrow & Stein, 2009; Luzón & Pérez-Llantada,
2019) and thus, it is essential to account for digital medium in our assessment framework for DMC. In the next section, we documented
how our preliminary framework was applied and refined in classroom-based practice.

4. Stage two: testing the multi-layered multimodal genre assessment framework

To lend empirical basis to test the theory-driven framework, the stage two of the study addresses the following research questions:
How did Chinese L2 writing teachers assess DMC with the preliminary multi-layered framework (Table 1) and what challenges and
practical issues did they encounter?

4.1. Context and the DMC projects

The context for stage two of the study was an English for general and academic purpose course called College English (CE) at a
technology-rich university in mainland China. Apart from reading, listening, speaking, and translating (between English and students’
L1, Chinese), L2 writing is an essential component of the course. To facilitate CE teaching, large-scale computer labs have been
established as instructional venues within which each student is provided a desktop with Internet bandwidth for self-paced language
learning. A survey conducted by the university also shows that the rate of student ownership of digital smartphones and laptop with in-
built cameras and recording devices amounts to 98 %. The course caters to non-English major students of various Bachelor programs,
including computer science, engineering, management, and biology. Given students’ access to digital tools and their diverse disci­
plinary backgrounds, the head of the CE department in the researched site contacted the first author, who then proposed DMC as one
pedagogic innovation to L2 writing. A collaborative action research team was then set up, including the first researcher, the head, and
nine other teachers who joined on an informed and voluntary basis.

Table 1
A multi-layered framework for DMC assessment.
Layer name Description and examples

Base layer (modal feature and selection)
Layout layer (modal feature and
combination)
Navigational layer (navigational and
move structure)
Rhetorical layer (rhetorical functions,
relations)
Base units: sentences, captions, images, audio, video clip, color
Group the base units together based on hierarchical structure, typographical/visual features, and placement of layout
units, which refer to headings, captions, photos, and other semiotic modes
Base units and layout units with navigational and move structure
Rhetorical relations holding between rhetorical segments, which mean rhetorical segments with rhetorical relations/
functions

4
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Following a project-based learning approach and Hafner and Ho’s (2020) process-oriented model, the DMC projects were
implemented by the participating teachers in their respective classrooms with their own decisions over the number, topics, and forms
of digital compositions. Throughout the process, the teachers engaged their students with a range of learning activities, including but
were not limited to reading, brainstorming, script writing, performing, editing, sharing, and reflecting. Different from previous studies
where students’ multimodal production was normally confined to one particular genre (such as documentary in Hafner & Ho, 2020), to
cater for students’ diverse interests, the teachers in our study gave full autonomy to students, who were from different academic
disciplines and guided and encouraged to compose a range of digital multimodal compositions, including videos, podcasts and posters
on a variety of self-selected genres, including narratives, documentaries, argumentations, and advertisements. To help the teachers
understand better the notion of genre, we referred to Hyland’s (2007) genre pedagogy in L2 writing and operationalized the notion of
genre in DMC as abstract, socially recognized ways of using semiotic resources that include but are not limited to languages, images,
and video clips. This means that as digital multimodal composers, students are expected to anticipate how their multimodal com­
positions (e.g., video) may be interpreted by target audiences based on their previous experiences of viewing and interpreting similar
compositions in certain communities and social contexts. We also discussed with the teachers that some genres in DMC can be more
elemental and some can be more macro. Elemental genres in DMC can be characterised as rhetorical patterns such as narratives,
arguments or expositions and these elemental genres can be combined into more complex everyday macro genres such as docu­
mentaries, drama, romance, or thrillers.
A sample assignment prompt can be seen in Appendix A. Aligning with a genre-based pedagogy, when preparing students for DMC
in the pre-designing stage, the teachers displayed sample videos of three different genres to students, including narrative, docu­
mentaries and argumentation. These sample videos included two videos authored by students in the first author’s previous research
projects and two videos produced by non-professional amateurs and then downloaded by the teachers from the Internet. The teachers
then engaged their students with analysis of intended purpose, structure of layout and rhetoric and multimodal expressions in the
selected sample videos. After sample analysis, the teachers engaged their students with guided multimodal composing through a range
of workshops on script writing, storyboarding and editing. Students were then given time to compose on their self-selected genres for
their DMC. The generic variations manifested by student works posed a significant challenge to the teachers when it came to
assessment. The first author then offered the preliminary assessment framework (Table 1) as part of our consultation materials to the
teachers. The components of the framework were explained to the teachers in detail. As researchers, we had the least intention to
impose the framework on the teachers, who were encouraged to comment on and adapt/implement the framework in situ according to
their own needs and preferences. The first author then observed the teachers’ response to the framework in their assessment practices.

4.2. Participants

While our collaborative action research started with 10 teachers, for this study we recruited five teachers (Zhao, Qian, Sun, Li, Zou,
all pseudonyms) who participated in the whole research process and were willing to share their DMC assessment experiences. The five
teachers, two male and three female, are all native Chinese – one with a PhD degree in education and the rest with a Master’s degree
each in applied linguistics. The teachers had an average 10.5 years of experience in CE teaching. They were keen on using technologies
in teaching and they had various experience of engaging students with video/audio recording of scripted performances. Their
experience with assessment of DMC was limited to the evaluation of language use in student-authored digital multimodal compositions
(e.g., videos, slides); hence how to assess DMC in a comprehensive manner presented a challenge. The collaborative action research
with the five teachers lasted 18 weeks, with three DMC projects implemented by the five teachers in their respective CE classes. The
implementation of each DMC project spanned approximately five weeks.

4.3. Data collection and analysis

This study draws on three main types of data gathered from the five participating teachers over 18 weeks: semi-structured in­
terviews, observations, and student-authored digital multimodal compositions selected by the teachers. In terms of interview, each
teacher was interviewed three times, with each conducted immediately by the first author after the completion of one DMC unit in their
respective classes. Each interview began with general questions about how the teachers evaluated DMC and the extent to which they
found the preliminary genre-based framework helpful and why. The teachers were also asked to talk about any challenges they
encountered and the changes, if any, that they made to the framework provided. Each interview, which lasted around 45 min, was
conducted in Chinese and audiorecorded.
At the end of each DMC project, a regular meeting among the five teachers was conducted and in total three such meetings were
held, focusing on how the teachers shared and exchanged their experiences of assessing DMC. These meetings were audiorecorded,
with observation notes made. The in-class feedback and sharing sessions led by the teachers were also observed, focusing on whether
and how the teachers commented on and evaluated students’ works with reference to the preliminary framework. Three such sessions
(one session per DMC project) were observed and audiorecorded with fieldnotes taken for each teacher throughout the research period.
At the semester end, each of the participating teachers was asked to select one student digital multimodal composition (e.g., videos)
for a stimulated recall interview. They were invited to talk about how they evaluated DMC with reference to the preliminary genre-
based framework. All the five teachers turned out to select the one that received the highest mark. Each teacher was also asked to share
individually any perceived challenges or issues regarding the framework for assessing DMC. These dialogues were also audiorecorded.
The analysis was informed by a qualitative and interpretive paradigm (Miles & Huberman, 1994). All the dialogical data were
transcribed verbatim and after accuracy check, the transcripts were imported into NVivo 12.0 for open coding. The first author

5
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

reviewed the transcripts multiple times and a tentative coding scheme emerged from the data after a process of constant comparisons.
For instance, we used “base unit” and “layout” as codes to categorize themes that emerged from the data. The naming of the major
themes within the scheme was informed by, but not limited to, the four layers of the preliminary framework. For instance, in addition
to the base, layout, rhetorical, and navigational layers, the teachers also suggested a need to add “purpose” as the core for the four
layers.
Drawing on the finding, the first author then had a discussion with the second author about refining the preliminary framework by
adding “purpose” as one additional layer. Based on the scheme and discussion, a refined genre-based model was developed with
reference to the teachers’ experiences and perceptions. Rubrics further developed by the teachers along each layer were integrated into
the model as subthemes. For instance, in the navigation layer, the teachers designed a table (see Table 4) summarizing common stages/
moves to be included in students’ multimodal compositions. By integrating these practice-based and teacher-initiated sub-themes, the
preliminary theory-driven framework was further developed into a refined genre-based model (see Fig. 6 in the discussion section) for
DMC assessment. The model and analysis were then double checked by the third author in order to enhance the trustworthiness of the
analysis.
In the following section, the themes that emerged from the data in response to the research questions are first presented, followed
by a discussion of the refined genre-based model for DMC assessment.

4.4. Findings

In general, all the five teachers found the four-layered preliminary framework (Table 1) “useful” in helping them specify “what and
how to assess” students’ DMC in a “systematic” manner (3rd interview-Zou). This section first presents the teachers’ overall assessment
practice, followed by their layer-specific assessment practice as well as the practical issues and challenges reported by the teachers.

4.4.1. An overview of the teachers’ DMC assessment practice

Overall, the teachers assessed students’ DMC in three main ways with reference to the framework. First, in the predesign and design
stages, the five teachers invited their students to present their DMC plans and storyboards in class for peer and teacher feedback. It was
at these stages that the five teachers found it necessary to direct students’ attention to their intended purposes and audiences of DMC,
which appeared missing in the preliminary framework (this is further presented as one practical issue reported by the teachers in the
following sections). For instance, Zhao suggested “it would be aimless if the students do not know clearly for whom and for what
purposes they are collecting, producing and combining all the semiotic resources.” (1st interview). To help students specify their
intended purposes and audiences, the teachers often included in their formative oral feedback one sentence with blanks for their
students to fill in: My DMC is to ___(inform, persuade) ___ (whom?) that ___(what is the central idea?). Through such feedback, the five
teachers highlighted the importance of evaluating multimodal composing across the four layers (base, layout, navigation, rhetoric) as
goal-directed and audience-oriented actions. Second, in the sharing stage, a video sharing session was conducted for students to play
their videos in class for open questions and feedback. The student-authored videos were assessed by the teachers with a rubric (Table 2)

Table 2
A genre-based rubric for assessing DMC.
Outstanding Good Satisfactory Failed

Purpose Able to present a clear idea with a
clear purpose (e.g., to inform,
persuade) for a specified
audience.
Base Units Able to include an appropriate
variety of base units of
outstanding quality, e.g., writing,
voiceover, soundtrack, video,
image, to the intended purpose
and audience.
Layout Able to group various base units
into a creative and interesting
layout entity with consistent
layout structure and appropriate
spatial relationships.
Navigation Able to present information in a
clearly organized and staged way,
using effective signposting with
an effectively organized body that
clearly follows conventions.
Rhetoric Able to organize an appropriate
variety of base units with clearly
identifiable and effective
rhetorical structures and
relationships that support the
overall purpose well.
Able to present idea with a
somewhat clear purpose (e.g.,
to inform, persuade) for a
specified audience.
Able to include a good variety of
base units of good quality, e.g.,
writing, voiceover, soundtrack,
video, image, to the intended
purpose and audience.
Able to group base units into a
good layout entity with good
layout structure and spatial
relationships.
Able to present information in
an organized and staged way,
using good signposting with an
organized body that follows
conventions.
Able to organize a variety of
base units with identifiable and
good rhetorical structures and
relationships that support the
overall purpose well.
Able to present idea with a
moderately clear purpose (e.g., to
inform, persuade) for somewhat
specified audience.
Able to include some base units of
satisfactory quality, e.g., writing,
voiceover, soundtrack, video,
image., to the intended purpose
and audience.
Able to group various base units
into a satisfactory layout entity
with satisfactory layout structure
and spatial relationships.
Able to present information in a
moderately organized and staged
way, using signposting with a
moderately organized body that
follows conventions.
Able to organize base units with
satisfactory rhetorical structures
and relationships that support the
overall purpose moderately well.
Some base units (e.g., visuals)
might appear inappropriate.
Unable to present idea with a clear
purpose (e.g., to inform, persuade)
for a specified audience.
Unable to include a variety of base
units of satisfactory quality, e.g.,
writing, voiceover, soundtrack,
video, image, to the intended
purpose and audience.
Unable to group various base units
into a satisfactory layout entity
with satisfactory layout structure
and spatial relationships.
Unable to present information in a
moderately organized and staged
way, using signposting with a
moderately organized body that
follows conventions.
Unable to organize base units with
satisfactory rhetorical structures
and relationships in support of the
overall purpose. Some base units
(e.g., visuals) are inappropriate.

6
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

they developed together for the task (The teachers’ layer-specific assessment practice is presented in the following section). A holistic
score was given by the teachers, who then referred to the analytical dimensions in the rubric for the justification of their score. Instead
of merely examining videos per se, the teachers highlighted students’ multimodal communicative competences in orchestrating a
range of semiotic modes for effective expression of ideas and content towards intended purposes and audiences. Then in the reflecting
stage, the five teachers invited their students to submit a written statement of goals and choices where they were invited to reflect on
their composing purposes, target audience, genre types, semiotic selections and combinations throughout the DMC process. Through
such reflections, the teachers guided their students to reflect on the appropriateness of multimodal selections and combinations to the
intended audiences and rhetorical purposes of DMC. In this way, the five teachers evaluated both the DMC process and the final
product (i.e., videos and base units such as script, image, and voiceover).
The rubric was co-developed by the five teachers and the researchers with reference to the preliminary framework. The rubric
comprises five categories of equal weighting, with four bands for each category, including outstanding, good, satisfactory and fail.
With the rubric, the five teachers would normally first identify the genre type of students’ DMC by examining the intended purpose and
audience, followed by a close examination of whether and how the multimodal selection and combination across the layers would
serve the overall purpose in ways that are appropriate to any genre-specific conventions. Specifically, the five teachers would specify
students’ selection of base units (e.g., writing, voiceover, visual) and examine whether and how these base units were grouped into
coherent visual clustering (i.e., layout grouping and spatial structure) on the screen with meaningful rhetoric clustering (i.e., effective
rhetoric structure and relationships) that supports the overall purposes for the intended audiences. Similar to the teachers in Hafner
and Ho’s (2020) study, while the five teachers emphasized creativity and novelty in students’ orchestration of multiple semiotic modes
for idea expression in their digital multimodal compositions, there was also an explicit expectation for the students to follow genre
conventions. This was evident in one assessment explanation (see Fig. 1) given by the teachers to their students in class regarding
students’ composing of narratives, which specifies clearly the six basic elements (time, place, person, cause, process, ending) to be
included and the importance for the narratives to be appealing and inspiring to the intended audience.
To achieve consistent scoring with the rubric, the teachers shared how they graded student-authored videos with the rubric and
gave feedback to each other in the third regular meeting. The meeting served as a standardization session for the teachers to clarify
what they considered “outstanding” or “good”, for instance, in layout and rhetorics. Their layer-specific assessment practice is pre­
sented in the next section.

4.4.2. Layer-specific assessment practice

The base layer
At the base layer, the five teachers reported that when assessing student-authored digital multimodal compositions, they would first
identify how many types of base units (e.g., video, image, soundtrack, on-screen caption, on-screen narration/voiceover) had been
employed, followed by a quality evaluation of each type. The base units of scripted caption and voiceover were more valued by the
teachers than other units, although such orientation can be critiqued as the language teachers’ biased preference of linguistic mode
over other semiotic modes (Jiang et al., 2019). The base units (e.g., images, video clips) that were originally produced/created by
students were also given more weights than those downloaded or borrowed from the internet or other sources. This was evident when
Zhao explained her evaluation guidelines in class:
“…You can use whatever you like to express yourself in your multimodal compositions, although as English teachers I would
expect you to always include your writing on screen. For copyright concerns, you are encouraged to use photos or videos that
you create on your own. If you can use clear photos or videos of high resolution, that is better. Your voiceover should be clearly
scripted and you are expected to minimize background noises for better vocal quality…” (Classroom observation-Zhao)
The other four teachers echoed that they would also value the vocal quality of audio and soundtrack and the resolution of image
and video. For language use in writing and voiceover, the teachers mainly attended to accuracy, fluency, and diversity in lexis and
syntax. The teachers appeared to be more tolerant of the physical quality of audio and video than the students’ language use, which
was related to their awareness of the physical and production constraints (e.g., lack of convenient access to recording studios or
equipment of professional standard) often experienced by their students during DMC. The teachers’ assessment practice at this layer
can be summarized in Table 3.

The layout layer

In addition to specifying the individual elements and base units, the teachers also found it necessary to attend to the typographical
or visual configurations of the base units on screen. Specifically, the teachers attended to the layout grouping, spatial relationships, and
other formatting properties (e.g., positioning and color of on-screen written captions) among various base units in student-authored
digital multimodal compositions. In order to facilitate students’ visual clustering during the process of DMC, Qian shared a layout
planning table she designed (Table 4) in the first meeting with the other four teachers, who found it helpful to guide and evaluate
students’ visual configuration and designs.

Fig. 1. One genre-specific assessment requirement (translated as: The six elements in narrative/stories (time, place, person, cause, process, ending) should
be complete; the story should be appealing and inspiring to the intended audiences).

7
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Table 3
Assessment at the Base layer.
Types of base units
• Written captions Quality of base units • Accuracy & diversity in grammar/lexis
• Voiceover • Fluency, pronunication, & articulation
• Soundtrack • Vocal quality, tempo, pitch, no noise
• Images/figures/drawing • Clarity, originally created/ borrowed
• Video clips • Resolution, originally created/ borrowed

Table 4
Layout planning.
Timeline
Track
Scene 1 Scene 2 Scene 3 …

Video/image
Transition
Audio/voiceover
Soundtrack/music
Written caption

As explained by Qian, student-authored multimodal videos can be divided into a series of scenes, the layout clustering of which
comprising configurations of base units along five tracks (i.e., the video/image track where students place videos/images they created
or downloaded, the transition track where students decide how their videos transit from one scene to another, the audio/voiceover
track where students integrate their audio-recorded narration, the soundtrack where students include music or sounds of their pref­
erence, and the scripted caption track where students decide how their writing would appear on the screen). In terms of grading, the
five teachers agreed that they expected their students to be consistent in layout clustering. For instance, Sun explained how she
evaluated one student-authored video in the second meeting:
In this video, the student is talking about AIDS situation in China. On the screen we could see his base units include images and
on-screen captions he wrote. We can also hear his voiceover, coupled with background music. No special transition effect was
used. What appears problematic to me is the sudden change of color and positioning of the two continuous on-screen captions
(see Fig. 2). This appears quite incoherent to me, although the student may prefer this random shift as his personal style… The
use of image as a base unit here also appears problematic because the student is describing a situation in China but the people in
the images do not look like Chinese...(2nd-regular-meeting, Sun)

It is clear that apart from spatial placements of various base units, modal interactions in ways of concurrence, complementarity, and
divergence (Jones & Hafner, 2012) were also considered by the five teachers when evaluating students’ visual configurations of
multiple base units. In addition, the five teachers suggested that from time to time, they would be amazed at some students’ originality
and talent in using special transition effects (see Fig. 3, where a picture-in-picture technique was applied to indicate transition between
two scenes), based on which the teachers would give extra credits. As Li stated in her interview: “Such transition effect exhibits the
student’s creativity in connecting two video clips into a coherent whole. So I will give him extra points for this creativity and talent.”

The navigational layer

At the navigational layer, the five teachers expected their students to construct their digital multimodal compositions with clearly
identifiable and genre-specific stages and signposts in order to facilitate readers’/consumers’ navigation and understanding. These
were evident when the teachers explained their evaluation standard in class:
“You normally need a title when you write something, right? It is important for you to have a title screen in your videos. Similar
to the title page of your written assignments, you can put down the name of authors and the title of your video stories. Then you

Fig. 2. Change of color and positioning of two continuous on-screen written captions.

8
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Fig. 3. Special effects (the snapshot in the middle) at the transition track.

can divide the main body of your videos into several parts with some clear signposts. You should also include an ending screen,
on which you give credits to any references, or sources that you downloaded from the internet.” (Classroom observation-Sun)
“Your video composing can be of scientific documentaries, narratives of stories, or argumentation through which you share your
personal viewpoints or arguments. Aligning with the scientific discourse, you can follow the introduction-method-finding-
discussion-conclusion structure for your scientific documentaries. For the narratives, it is important for you to include six
essential parts, namely time-setting-people-orientation-process-consequence. For argumentation, you should have a central
point with various supporting points or evidence…” (Classroom observation-Li)
The teachers then collaboratively constructed a table of stages (see Table 5) they would normally expect when assessing students’
DMC.
Following the stage guidelines set out in Table 5, the five teachers agreed to nominate one student-authored video as an exemplar of
narratives for its clearly staged narration from the beginning to the end. The screenshots that represent some of the stages in the
narration were presented in Fig. 4.
Fig. 4, from left to right, displays the screenshots of the title screen and the 1st episode, followed by the 1st signpost (i.e., a black
transitional screen with “12 months ago” as the written caption), 2nd and 3rd episode, and ending screen (screenshots for episode 4th-
6th are omitted due to space limit). This narrative recounts a first-person story comprising six episodes with a clear chronological
order, within each the essential parts (e.g., time, setting, process, consequence) of narratives are presented in a clearly recognizable
manner.

The rhetorical layer

At the rhetorical layer, the teachers attended to meaning relations between and within base units in student-authored multimodal
compositions. As Zou and Li stated in the second meeting:
Students normally combine their writing with several other elements during DMC and I would expect all combinations to be
meaningful. If the combinations can signal some meaningful relationships or follow some rhetorical organizations, I would
usually award a higher score. (2nd meeting-Zou)
Some students combine different resources in a rather arbitrary manner and manifest a poor awareness of rhetorical relations.
They tend to include images and background music out of their own preference or in order to show their coolness. Such
combinations turn out to be not-so-sensical and I normally give a lower grade… (2nd meeting-Li)
The five teachers agreed that the RST relations offered them a good point of reference when evaluating and guiding students’
construction of rhetorical relations and structures in their digital multimodal compositions. For instance, in his in-class feedback on
one student-authored video, Zou commended on the students’ strategic use of a multi-nuclear relation to list three pieces of evidence in
support of a claim as well as several symmetric relations for the purposes of offering evidence and elaboration. To facilitate his
comment, Zou displayed his hand-drawn rhetorical structures in class (see Fig. 5).
The student-authored video described by Zou can be regarded as an argumentation, within which the students advocated the idea
that university students should cultivate responsible behaviors. To support the idea, the students included evenly three episodes on
respecting diverse opinions, avoiding overconsumption and internet fraud. The students parodied the popular genre of news report,
with one student acting as a news program host reporting both problems and solutions in each episode. The other students then
collaboratively acted out specific scenes and the performances were video recorded and edited to offer evidence or elaboration to the
main points reported in their imagined news program. In particular, Zou appreciated the students’ ability to display humor and
creativity by layering verbal and written captions (through which student A is talking about his love for a “beauty” on the Internet, see

Table 5
Stages in various genres of digital multimodal compositions.
Genre Stages (“^” means “followed by”)

Narratives title screen^ time^ setting^ people^ orientation^ process^ consequence ^ ending screen
Scientific documentaries title screen^ introduction^ method^ findings^ discussion^ conclusion^ ending screen
Argumentation title screen^ central point^ subpoint 1^ evidence^ subpoint 2^ evidence^ subpoint 3^ evidence^ conclusion^ ending screen

9
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Fig. 4. Screenshots of screens that indicate stages in narratives.

Fig. 5. The rhetorical structure and relations drawn by Zou.

Fig. 6) with images that diverge significantly from the linguistic meanings (e.g., the image that depicts the “beauty” on the right side of
Fig. 5). Such a contrast relation, together with the lines, is displayed in Fig. 6.

4.4.3. Practical issues and challenges

Throughout the process of assessing DMC with the preliminary four-layered framework, the five teachers gradually realized a need
to include DMC purpose as an additional layer, with three suggested adding DMC purpose as the core layer with which the other four
layers (i.e., base unit, layout, navigation, rhetoric) can be associated and interconnected. The five teachers had the following discussion
in the third meeting:
Zhao: The four-layered framework is helpful but I feel a need to also include purpose, the purpose of composing…
Qian: Yes. Purpose is the starting point. With purpose, students can then decide what base units to include and what rhetorical
patterns they may need.
Sun: Normally, students have two main types of purpose: to inform or narrate something and to argue or persuade. Some
students’ compositions are really messy because from the very beginning, they do not have a clear purpose and audience…
Zou: Exactly, we’d better let students know that purpose is really the core.
(3rd meeting-observation)

10
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Fig. 6. A rhetorical relation of “contrast” in student video.

The other issues reported by the five teachers included concerns over the legitimate evidence of learning and teachers’ confidence
in DMC assessment. In the beginning, when the preliminary framework was introduced to the teachers, two had reservations about
taking writing as only one of the multiple base units, which appeared to diminish the importance of writing as legitimate evidence of
learning during DMC. For instance, Li stated in the first meeting:
The framework appears quite comprehensive. The problem appears to be that writing seems to take a small part as it is only one
of the base units, right? (1st meeting-Li)
Such concern disappeared by the end of the semester when the five teachers had more experience in using the framework. In his
final interview, Li stated:
I used to feel that the framework focused not that much on writing, which seems to be reduced to a base unit at only one of the
layers. Then after one semester’s work, I feel many aspects of writing in relation to genre, organization and coherence are still
relevant. The framework helps me see better now how writing can be orchestrated together with other semiotic modes and
resources. (Interview-Li)
The five teachers also reported a need to further develop competence in DMC assessment at the layout and rhetorical layers:
I am still not very confident to evaluate the layout as it involves several tracks, which are not what I am familiar with. Although I
can draw on my own media experience, I am not totally sure about the modal interactions across the tracks of video, audio, and
soundtrack. I guess this is the point that I should learn more from students. (Interview-Qian)
The rhetorical relations between words and other modes such as images and soundtrack appear quite a new thing to me as I have
been a language teacher for a long time. Students, however, appear quite capable of using, for example, images or soundtrack to
indicate shift in narration or to strike a contrast in meaning… (Interview-Sun)
While the five teachers agreed that their media experience and genre knowledge can be of some help when evaluating DMC, there
was a felt need to learn from students in terms of modal interactions and layering strategies as well as a need for continuing pro­
fessional development to enhance their digital assessment literacy.

5. Discussion

Informed by the GeM theory (Bateman, 2008), this study first proposed a preliminary four-layered framework for assessing DMC in
the context of L2/EFL writing. Then grounded in the practice of DMC assessment in authentic classrooms, the study tested and refined

11
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

the preliminary framework with reference to teacher experiences and perspectives. Based on the integration of theory and practice and
the findings of the study, a genre-based model for assessing DMC is presented as follows (see Fig. 7).
As displayed in Fig. 7, the genre-based model centers around purpose, which can be taken as an overarching guide for teachers to
evaluate whether and how students’ composing across the other four layers (i.e., base, layout, navigation, rhetoric) help achieve the
overall communicative purposes and goals (e.g., to inform, to persuade). As such, the notion of genre in the model can be theorized as
purpose/goal-directed and audience-oriented social action, which is then realized through multiple semiotic choices and orchestration
within and across multiple layers and strata. As manifested by the five teachers’ assessment practice in the study, at the base unit layer,
teachers can evaluate whether the base units (e.g., writing, script, image) employed by students are of certain diversity and reasonable
quality. At the layout layer, teachers can assess how students arrange and layer multiple base units into consistent visual grouping and
configurations on screen (or page) by taking into account potential spatial relationships and formatting properties (e.g., position, color,
size) of various base units. It should be noted that students’ DMC at the base and layout layers is subject to physical and production
constraints (Bateman, 2008). Thus, when evaluating DMC, teachers can be tolerant about the material quality of student-authored
digital multimodal compositions because after all, students are not professional video producers or poster designers and they may
not necessarily have access to high-quality facilities for the production of high-quality image or video. Thereby the assessment focus
can be placed on whether and how students make the best use of what they can access in terms of selecting and combining base units
for their DMC.
Assessing DMC at the navigation and rhetorical layers focuses primarily on meanings. At the navigation layer, teachers can assess
whether and how students organize their content in genre-appropriate stages/moves and orders (e.g., topic order, chronological
order). In this study, special recognition was also given by the teachers to acknowledge students’ multimodal competences in capi­
talizing on the affordance of different modes to facilitate targeted consumption and understanding of their videos. This was evident
when the teachers gave extra credits to students’ ingenious use of soundtrack (e.g., the sound of raindrops), color (e.g., color black in
the transitional screen), or on-screen font size as signposts and transitions in their digital multimodal compositions. At the rhetorical
layer, teachers can attend to whether and how students construct logical rhetorical interrelationships within and across multiple base
units. The moves and rhetorical relations are to facilitate target readers’/consumers’ understandings and they can be regarded as
consumption constraints (Bateman, 2008) on students’ DMC. As such, it becomes possible for teachers to evaluate students’ read­
er/audience (or consumer) awareness by evaluating whether and how students employ genre-specific stages/moves and rhetorical
relations in their digital multimodal compositions.
Altogether, the study adds to the extant literature on assessing DMC and contributes an empirically evidenced genre-based model to
further bridge the gap between teaching and assessing DMC in L2 writing. The model complements the element-based rubrics (Hung
et al., 2012) by specifying further the rhetorical relations between language and other semiotic modes during DMC and how such
relations can be accounted for at the layers of rhetoric and base unit. The model also complements the process-based model (Hafner &
Ho, 2020) by including genres other than documentaries such as narratives and argumentations into assessment consideration.
Different from the previously reported element-based rubrics (Hung et al., 2012) and the process-oriented model (Hafner & Ho, 2020),
our study accounts for potential variations in the genres of student-authored digital multimodal compositions in DMC assessment.
Aligning with previous works (e.g., Belcher, 2017; Hafner, 2015; Smith, Pacheco, & de Almeida, 2017; Jiang, Yang & Yu, 2020; Jiang,
2021) on the agentic and motivated nature of students’ multimodal design, our model highlights that students’ DMC should be better
conceived as a dynamic compromise between personal interest and the constraints of material and targeted consumption. As man­
ifested by the study (e.g., Fig. 3), it is at the intersection of DMC purpose, personal motivation, and the constraints of material,
production and consumption that students’ multimodal choices with apt resources are achieved and concretized. In light of the
commonly accepted importance of making appropriate multimodal choices when constructing digital ensembles (Hafner, 2015; Yi
et al., 2020), an additional contribution of our study is the observation that multimodal orchestration is a multifaceted activity that
occurs within and across at least five major layers rather than a one-off monolithic activity. Such layer-specific multimodal choices and
combinations not only make it more manageable for teachers to evaluate students’ multimodal design, but also specify how apt modal
choices and interactions in students’ compositions can be better visualized and scaffolded.
In response to previous reports (e.g., Yi & Choi, 2015) on teachers’ concerns over the links between DMC and writing, another
benefit of using the multilayered genre-based model (Fig. 7) for assessing DMC, as manifested by the five teachers in the study, is a
better possibility for L2 teachers to see the roles of writing-related knowledge during DMC. Such knowledge includes but is not limited
to accuracy in language use, diversity in lexis, generic stages/moves, and rhetorical relations. This explained why the genre-based
model was not resisted by the five L2 teachers, although two did have some reservations about the place of writing during DMC in
the beginning stage. Moreover, attending to rhetorical relations between writing as a base unit and other base units (e.g., image,
soundtrack, bodily performance) makes it possible for L2 teachers to evaluate students’ multimodal communicative competences by
examining how students may combine writing (or language) with other semiotic modes (see the “contrast” relation in Fig. 6) in their
digital multimodal compositions. This finding further specifies the role of language and writing in DMC assessment, something that has
been critiqued as being not focused on in previous element-based rubrics (Hung et al., 2012) for multimodal assessment. This spec­
ification is significant because it indicates that for L2 writing teachers to build up their metalanguage of multimodality, which has been
described as something essential but challenging when assessing DMC (Yi & Choi, 2015; Jiang et al., 2019; Jiang, Yu & Zhao, 2021;
Jiang & Luk, 2016), developing knowledge of rhetorical relations and structures at the rhetorical layer may be a good place to start.
Aligning with Hafner and Ho (2020) and Kress and Van Leeuwen (2006), our study lends further support to the idea that it is
possible to expand genre from print-based writing to DMC. In relation to the GeM framework (Bateman, 2008; Hiippala, 2017), our
study gives credence to its applicability in the context of DMC. Nevertheless, situating within one university-based EFL curriculum and
the experiences of five teachers, our findings should be interpreted with caution as they may not be generalized across contexts. Given

12
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Fig. 7. A genre-based model for assessing DMC.

the limited time span of the study, the long-term impact of the genre-based assessment model upon L2 writing remains to be further
explored. As the students in the teacher participants’ classes mostly produced videos as a form of DMC, whether and how our
genre-based assessment model can be expanded to other forms of DMC is worthy of future research attention. This study focuses on the
teachers; future research can examine students’ response to the genre-based model and how assessment and instruction informed by
the model may impact their learning. In terms of pedagogy, further research can also explore whether and how genre-based feedback
and instruction can be better planned and provided in support of students’ DMC.

6. Conclusion

In response to the challenge of assessing DMC in the context of L2 writing, this study complements previous element-based rubrics
(Hung et al., 2012) and process-oriented approach (Hafner & Ho, 2020) by developing and refining a theory-driven and
empirically-evidenced genre-based model. As manifested by the findings, our model guides teachers to evaluate student-authored
digital multimodal compositions as purpose-directed and audience-oriented social actions to be constructed with apt multimodal
choices within and across five major layers, i.e., purpose, base units, layout, navigation, and rhetoric. In addition to specifying what apt
multimodal choices may look like across the layers, our model can also help teachers visualize modal interactions between writing and
other semiotic modes. Thus, the model can help train L2 teachers how to approach the assessment of DMC in general. The efficacy of
the genre-based model, however, merits further research across multimodal composing forms and contexts. Teachers also need to
receive sufficient training to understand and implement the model geared towards specific DMC tasks.

Declaration of Competing Interest

The authors declare that they have no conflicts of interest.

Acknowledgements

We would like to acknowledge the support for the study from Faculty Research Fund (EA210397), Faculty of Educatition, Uni­
versity of Hong Kong.

Appendix A. A sample assignment prompt

2019-2020-2 Assignment requirement and prompt

Compose with 1–2 and produce a video, or poster or other forms of multimodal texts. The topics can be selected from the 12 themes
in the textbook. You can also self select a topic.

13
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

The video should be of about three minutes. The maximum is five minutes.
In terms of genre, you can select one from the following: scientific documentaries, narratives, argumentation (share your opinions
on some topics or issues), or any other genres that you feel appropriate.

Appendix B. Interview questions

1 How did you evaluate students’ DMC?

2 Did you find the preliminary genre-based framework helpful? Why or why not?
3 In what ways you find the framework helpful for your evaluation? Can you give one example?
4 Have you encountered any challenges in your use of the framework? If yes, what were the challenges and how did you respond?
5 Did you make any changes to the preliminary framework? If yes, what were those changes?
6 In what ways can the preliminary framework be improved? Why?

References

Askehave, I., & Nielsen, A. E. (2005). Digital genres: A challenge to traditional genre theory. Information Technology and People, 18(2), 120–141. https://doi.org/
10.1108/09593840510601504
Bateman, J. (2008). Multimodality and genre: A foundation for the systematic analysis of multimodal documents. London: Palgrave Macmillan.
Bateman, J. (2014). Developing a GeM (genre and multimodality) model. In S. Norris, & C. D. Maier (Eds.), Interactions, images and texts: A reader in multimodality (pp.
25–36). Berlin and New York: De Gruyter Mouton.
Belcher, D. D. (2017). On becoming facilitators of multimodal composing and digital design. Journal of Second Language Writing, 38, 80–85. https://doi.org/10.1016/j.
jswl.2017.10.004
Burnett, R. E., Frazee, A., Hanggi, K., & Madden, A. (2014). A programmatic ecology of assessment: Using a common rubric to evaluate multimodal processes and
artifacts. Computers and Composition, 31, 53–66. https://doi.org/10.1016/j.compcom.2013.12.005
Elola, I., & Oskoz, A. (2017). Writing with 21st century social tools in the L2 classroom: New literacies, genres, and writing practices. Journal of Second Language
Writing. https://doi.org/10.1016/j.jslw.2017.04.002
Giltrow, J., & Stein, D. (2009). Genres in the internet. Amsterdam and Philadelphia: John Benjamins.
Hafner, C. A. (2015). Remix culture and English language teaching: The expression of learner voice in digital multimodal compositions. TESOL Quarterly, 49(3),
486–509. https://doi.org/10.1002/tesq.238
Hafner, C. A., & Ho, W. Y. J. (2020). Assessing digital multimodal composing in second language writing: Towards a process-based model. Journal of Second Language
Writing, 47, Article 100710. https://doi.org/10.1016/j.jslw.2020.100710
Hafner, C., & Miller, L. (2011). Fostering learner autonomy in English for science: A collaborative digital video project in a technological learning environment.
Language Learning & Technology, 15(3), 68–86.
Hafner, C., & Miller, L. (2019). English in the disciplines: A multidimensional model for ESP course design. New York: Routledge.
Hiippala, T. (2012). Reading paths and visual perception in multimodal research, psychology and brain sciences. Journal of Pragmatics, 44(3), 315–327. https://doi.
org/10.1016/j.pragma.2011.12.008
Hiippala, T. (2017). An overview of research within the Genre and Multimodality framework. Discourse Context & Media, 20, 276–284. https://doi.org/10.1016/j.
dcm.2017.05.004
Hung, H., Chiu, Y. J., & Yeh, H. C. (2012). Multimodal assessment of and for learning: A theory-driven design rubric. British Journal of Educational Technology. https://
doi.org/10.1111/j.1467-8535.2012.01337.x
Hyland, K. (2007). Genre pedagogy: Language, literacy and L2 writing instruction. Journal of Second Language Writing, 16, 148–164. https://doi.org/10.1016/j.
jslw.2007.07.005
Jiang, L., Yu, S., & Zhao, Y. (2021). Incorporating digital multimodal composing through collaborative action research: Challenges and coping strategies. Technology,
Pedagogy and Education, 1–17. https://doi.org/10.1080/1475939X.2021.1978534
Jiang, L. J. (2017). The affordances of digital multimodal composing for EFL learning. ELT Journal, 71(4), 413–422. https://doi.org/10.1093/elt/ccw098
Jiang, L. J. (2018). Digital multimodal composing and investment change in learners’ writing in English as a foreign language. Journal of Second Language Writing, 40,
60–72.
Jiang, L. (2021). Facilitating EFL students’ civic participation through digital multimodal composing. Language, Culture and Curriculum, 1–16. https://doi.org/
10.1080/07908318.2021.1942032
Jiang, L., & Luk, J. (2016). Multimodal composing as a learning activity in English classrooms: Inquiring into the sources of its motivational capacity. System, 59, 1–11.
https://doi.org/10.1016/j.system.2016.04.001
Jiang, L., & Ren, W. (2020). Digital multimodal composing in L2 learning: Ideologies and impact. Journal of Language, Identity & Education, 1–16. https://doi.org/
10.1080/15348458.2020.1753192
Jiang, L., Yang, M., & Yu, S. (2020). Chinese ethnic minority students’ investment in English learning empowered by digital multimodal composing. TESOL Quarterly,
54(4), 954–979. https://doi.org/10.1002/tesq.566
Jiang, L., Yu, S., & Zhao, Y. (2019). Teacher engagement with digital multimodal composing in a Chinese tertiary EFL curriculum. Language Teaching Research. https://
doi.org/10.1177/1362168819864975
Kim, Y., & Belcher, D. (2020). Multimodal composing and traditional essays: Linguistic performance and learner perceptions. RELC Journal, 51(1), 86–100. https://
doi.org/10.1177/0033688220906943
Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London and New York: Routledge.
Kress, G., & Van Leeuwen, T. (2006). Reading images: The grammar of visual design. London: Routledge.
Li, M. (2020). Multimodal pedagogy in TESOL teacher education: Students’ perspectives. System. , Article 102337. https://doi.org/10.1016/j.system.2020.102337
Luzón, M. J., & Pérez-Llantada, C. (2019). Connecting traditional and new genres: Trends and emerging themes. In M. J. Luzón, & C. Pérez-Llantada (Eds.), Science
communication on the internet: Old genres meet New genres (pp. 1–18). Amsterdam: John Benjamins.
Mann, W. C., & Thompson, S. A. (1988). Rhetorical structure theory: Toward a functional theory of text organization. Text, 8(3), 243–281.
Miles, M., & Huberman, A. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks: Sage Publications.
Miller, C. R. (1984). Genre as social action. Quarterly Journal of Speech, 70(2), 151–167.
Mills, K. A., & Exley, B. (2014). Time, space and text in the elementary school digital writing classroom. Written Communication, 1–36. https://doi.org/10.1177/
0741088314542757
New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60–92.

14
L. Jiang et al. Journal of Second Language Writing 57 (2022) 100869

Prior, P. A. (2009). From speech genres to mediated multimodal genre systems: Bahktin, Voloshinov, and the question of writing. In C. Bazerman, A. Bonini, &
D. Figueiredo (Eds.), Genre in a changing world (pp. 17–34). Fort Collins: The WAC Clearinghouse and Parlor Press.
Smith, B. E., & Shen, J. (2017). Scaffolding digital literacies for disciplinary learning: Adolescents collaboratively composing multimodal science fictions. Journal of
Adolescent & Adult Literacy, 61(1), 85–90. https://doi.org/10.1002/jaal.660
Smith, B. E., Pacheco, M. B., & de Almeida, C. R. (2017). Multimodal codemeshing: Bilingual adolescents’ processes composing across modes and languages. Journal of
Second Language Writing. https://doi.org/10.1016/j.jslw.2017.04.001
Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge, UK: Cambridge University Press.
Unsworth, L. (2007). Multiliteracies and multimodal text analysis in classroom work with children’s literature. In T. D. Royce, & W. L. Bowcher (Eds.), New directions
in the analysis of multimodal discourse (pp. 331–360). Mahwah, New Jersey, London: Lawrence Erlbaum Associates Publishers.
van Leeuwen, T. (2005). Introducing social semiotics. New York: Routledge.
Waller, R. (1987). The typographical contribution to language: Towards a model of typographic genres and their underlying structures. PhD thesis. Reading, U.K: Department
of Typography and Graphic Communication, University of Reading http://www.robwaller.org/RobWallerthesis87.pdf.
Wyatt-Smith, C., & Kimber, K. (2009). Working multimodally: Challenges for assessment. English Teaching-Practice and Critique, 8(3), 70–90. Retrieved from <Go to
ISI>://WOS:000273257700006.
Yi, Y., & Choi, J. (2015). Teachers’ views of multimodal practices in K-12 classrooms: Voices from teachers in the United States. TESOL Quarterly, 29(4), 838–847.
https://doi.org/10.1002/tesq.219
Yi, Y., King, N., & Safriani, A. (2017). Reconceptualizing assessment for digital multimodal literacy. TESOL Journal, 8(4), 878–885. https://doi.org/10.1002/tesj.354
Yi, Y., Shin, B. S., & Cimasko, T. (2020). Special issue: Multimodal composing in multilingual learning and teaching contexts. Journal of Second Language Writing, 47.
https://doi.org/10.1016/j.jslw.2020.100717
Zhang, M., Akoto, M., & Li, M. (2021). Digital multimodal composition in post-secondary L2 settings: A review of the empirical landscape. online first Computer
Assisted Language Learning. https://doi.org/10.1080/09588221.2021.1942068.

Dr. Lianjiang Jiang is an Assistant Professor at Faculty of Education, The University of Hong Kong. His research interests include second language writing, digital
multimodal composing, and new literacy studies. His publications have appeared in Language Teaching Research, System, ELT Journal, Journal of Second Language Writing,
Journal of Language, Identity, and Education, Computer Assisted Language Learning and TESOL Quarterly.

Dr. Shulin Yu is an Associate Professor at Faculty of Education, University of Macau, Macau SAR, China. His research interests include second language writing and
second language writing teacher education. His publications have appeared in Journal of Second Language Writing, Assessing Writing, Language Teaching Research, Applied
Linguistics Review, Assessment & Evaluation in Higher Education, Language Teaching, and TESOL Quarterly.

Prof. Icy Lee is a Professor in the Faculty of Education at The Chinese University of Hong Kong. Her publications have appeared in international journals such as
Assessing Writing, Journal of Second Language Writing, Language Teaching, Language Teaching Research, System, and TESOL Quarterly.

15