Computers & Education 173 (2021) 104271

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier.com/locate/compedu

Understanding feedback in online learning – A critical review and

metaphor analysis
Lasse X Jensen a, b, *, Margaret Bearman b, David Boud b, c, d
a
Department of Public Health, University of Copenhagen, Copenhagen, Denmark
b
Centre for Research in Assessment and Digital Learning, Deakin University, Melbourne, Australia
c
Faculty of Arts and Social Sciences, University of Technology Sydney, Sydney, Australia
d
Work and Learning Research Centre, Middlesex University, London, UK

A R T I C L E I N F O A B S T R A C T

Keywords: Technologies associated with online learning have led to many new feedback practices and
Online learning expanded the meaning of feedback beyond the traditional focus on instructor comments, but
Feedback conceptual work on online feedback has not followed. This paper investigates how online learning
Qualitative methods
researchers understand feedback’s role in teaching and learning, and discusses how these un­
Metaphor analysis
derstandings influence what research questions are asked, and what online feedback practices are
Critical review
Assessment for learning recommended.
Through a qualitative analysis of the language used about feedback in leading research jour­
nals, we identified six distinct understandings of feedback based on six dominant conceptual
metaphors. These are feedback is a treatment, feedback is a costly commodity, feedback is
coaching, feedback is a command, feedback is a dialogue, and feedback is a learner tool.
Each of these metaphors offers a coherent frame of entailments related to the roles and re­
sponsibilities of online instructors and online learners as well as some bigger assumptions about
what role feedback should play in online teaching and learning. A comparison with current
feedback research revealed that just two of the six metaphors align with the learner-centric
feedback practices that are increasingly considered appropriate among feedback researchers.
The paper discusses how the conceptualizations might reflect different challenges facing online
education.
The paper proposes that researchers interrogate their own conceptualizations to ensure that
they align with their beliefs about feedback and its role in the learning process. It suggests that a
more deliberate use of metaphors when conceptualizing feedback and online feedback practices is
necessary for clarity of communication and helpful for driving the work on feedback in online
learning forward.

1. Introduction

Feedback is a crucial yet challenging part of teaching and learning (Kluger & DeNisi, 1996; Sadler, 1989; Shute, 2008). In online
learning, which can be a remote and solitary activity, the role of feedback may be even more essential, because it is one of few processes

* Corresponding author. Department of Public Health, Faculty of Health and Medical Sciences, University of Copenhagen, Bartholinsgade 4, DK-
1356 Copenhagen K, Denmark.
E-mail address: lassej@sund.ku.dk (L.X. Jensen).

https://doi.org/10.1016/j.compedu.2021.104271
Received 18 December 2020; Received in revised form 28 March 2021; Accepted 2 July 2021
Available online 3 July 2021
0360-1315/© 2021 Published by Elsevier Ltd.
L.X. Jensen et al. Computers & Education 173 (2021) 104271

that connects individual learners to instructors and peers. Despite agreement on its importance, feedback’s role in online learning is not
clearly defined and there are many overlapping and sometimes contradictory understandings. The technologies and trace data that are
characteristic of many online learning environments have greatly expanded what we can consider in feedback (e.g. Pardo, 2017;
Tempelaar, Rienties, & Giesbers, 2015). While feedback once mostly implied teacher comments on student work, the rise of digital
learning has meant that it is now also used to describe many forms of automatically generated information. For example, online quizzes
with pre-set commentaries are often considered feedback (e.g. Förster, Weiser, & Maur, 2018; Maier, Wolf, & Randler, 2016). There is
therefore an urgent need to review how researchers in online education conceptualize feedback in order to ensure that both new
advances in technology and pedagogy can be appropriately integrated into online environments.
Using new technologies is often seen as a way to make feedback more effective. Some have focussed on using technology to
automate creation and transmission of feedback information in order to increase the amount of individual feedback that each learner
receives (Bälter, Enström, & Klingenberg, 2013; Pardo, Jovanovic, Dawson, Gašević, & Mirriahi, 2017). Other researchers have
focussed on improving the quality and timing of feedback comments that online instructors generate, in order to enhance their impact
(Alvarez, Espasa, & Guasch, 2012; Van der Kleij, Feskens, & Eggen, 2015). Despite substantial progress in this body of research, and the
many affordances of educational technologies, feedback often fails to deliver on its promises. One key barrier is that, as with
face-to-face modes, online learners often do not engage with the feedback information that they receive (Mensink & King, 2020;
Winstone, Bourne, Medland, Niculescu, & Rees, 2020).
Discussions of how to address the lack of student engagement with feedback often comes down to questions of how we concep­
tualize feedback. On the one hand, many researchers do not consider feedback a process for learners to participate in, but information
that is passed on to learners (Hattie & Timperley, 2007). Within this understanding, the main feedback challenge in online education is
not lack of learner involvement, but simply that each learner receives low quality, inappropriate or too little information. This is
associated with a research focus on identifying ways to fine-tune and automatize the composition and delivery of feedback input, while
paying much less attention to what the learner is doing. On the other hand, there is a shift among some feedback researchers towards a
new paradigm (Carless, 2015; Winstone & Carless, 2019) characterized by dialogic processes (Nicol, 2010) and a focus on student
agency (Evans, 2013; Van der Kleij, Adie, & Cumming, 2019). This reconceptualization has paved the way for more robust theories
about the nature and purpose of feedback, as well as interesting new work that examines the role of feedback in important phenomena
such as building evaluative judgement (e.g. Tai, Canny, Haines, & Molloy, 2015; Tai & Sevenhuysen, 2018), sustainable assessment
(Boud & Soler, 2016) and feedback literacy (Sutton, 2012). Although it is unclear how these ideas have played out in practice and
penetrated into broader research, discussions are particularly muted in the digital space, where research tends to be practically ori­
ented and less conceptual (Bennett & Oliver, 2011; Johnson, 2015, pp. 35–50).
To improve our understanding of the role of feedback in digital learning environments, we must start by identifying which implicit
and explicit conceptualizations of feedback are present in research on online learning. This is a necessary first step towards having a
robust debate about the different ways feedback in online learning is conceptualized and hence operationalised. Such a debate is useful
to sharpen our thinking in relation to a host of online teaching practices and course elements, including the use of peer review systems,
automated quizzes, learning analytics, and formative assessment. Mapping the way researchers conceptualize feedback in online
learning will also illuminate the different ways our, sometimes unconscious, conceptualizations shape the research questions we ask
and the online feedback practices we recommend.

1.1. Aims

In this paper, we identify and discuss the conceptualizations used by researchers when they explore feedback in online learning.
Inspired by conceptual metaphor theory (Lakoff & Johnson, 1980), we consider such conceptualizations to be inherently metaphorical,
and thus it is possible to identify and examine them through metaphor analysis.
Our approach combines metaphor analysis with a critical review methodology. A critical review is one where a subset of data is
critically examined for content (Grant & Booth, 2009; Norman & Eva, 2010, pp. 301–322). Our approach includes the following steps.
First, we create a dataset of metaphorical language about feedback found in research publications about online and digital learning; then
we interpret this metaphorical language through providing overarching conceptual metaphors based on the implied conceptualisations;
and finally, we discuss what these conceptual metaphors may reveal about researchers’ understandings of feedback and its role in
online teaching and learning. Our dataset, the metaphorical language used by researchers, is generated from recent research articles
published in leading online and digital learning journals.

2. Methods

We conduct our metaphor analysis against the backdrop of a constructivist epistemology, which emphasizes the connection be­
tween language and social reality. In accordance with standards for qualitative research we are concerned with the trustworthiness of
our claims (Holloway & Todres, 2003). To this end, this methods section describes the theoretical foundations for our work, situates the
study epistemologically, and details the steps undertaken in generating and analysing the dataset. This concern for trustworthiness also
entails recognizing the inherent subjectivity of our approach and seeking to manage this through our methods, for instance by having
two researchers independently code the data, and by reflexively considering our own positionality and recognizing that we, as
feedback researchers, bring certain preconceptions, experiences and opinions about feedback and digital education to the analysis.
Following the guidelines for qualitative studies proposed by Twining, Heller, Nussbaum, and Tsai (2017), we conduct and report
this as an inductive project, taking as our outset the analysis of data and introducing relevant previous research in the subsequent

2
L.X. Jensen et al. Computers & Education 173 (2021) 104271

discussion of findings.

2.1. Conceptual metaphor theory

Metaphor is not only a non-literal element of language used for aesthetic reasons, but also an inherent part of how we make sense of
the world (Lakoff & Johnson, 1999). To be able to conceptualize and talk about abstract concepts, the mind borrows the structure of
another, less complicated, concept. This is called a conceptual metaphor (Lakoff & Johnson, 1980). An example of such a conceptual
metaphor is Sfard’s famous argument that learning, a complex phenomenon, is often understood in terms of the less complex phe­
nomenon acquisition (Sfard, 1998). This LEARNING IS ACQUISITION metaphor structures our understanding of learning in a way that
makes expressions like gaining, acquiring or sharing knowledge meaningful. Although conceptual metaphors allow us to understand
abstract phenomena (e.g. learning) by way of less abstract phenomena (e.g. acquisition), they also limit how we understand them. This
limitation comes in the form of metaphorical entailments, that is, the metaphor profoundly influences the way we talk and think about
the phenomenon and its relation to other phenomena. As an example, the conceptual metaphor A UNIVERSITY IS A FACTORY enables
one coherent frame of understandings and expressions that consider standardization, competitiveness, automation, revenue growth
and cost cutting to be central success criteria for a university. When our thinking is framed by this metaphor, we might be blinded to
other possible understandings – and criteria for success – that can emerge when we see the university as e.g. a laboratory (curi­
osity-driven, testing out new ideas), or a family (shaping you as a person, life-long relation).
This showing/hiding function of metaphors and what they reveal about the understandings of the people using them, have made
metaphor analysis a popular form of qualitative inquiry (Todd & Low, 2010). Within education research, metaphor has been used as a
lens to analyse many phenomena, e.g. teacher identities (L. Thomas & Beauchamp, 2011) or doctoral education (Nye, Foskey, &
Edwards, 2014). Metaphor analysis can highlight the taken-for-granted assumptions, and help us understand how this influences our
practices. This paper uses the approach to investigate the phenomenon of feedback, as it is understood by online and digital learning
researchers.

2.2. Selecting papers for inclusion in study

Our dataset is a sample of the metaphorical language used in research literature that investigates feedback in online environments.
In line with other critical reviews (Ashwin, 2012; Bearman, Mahoney, Tai, Castanelli, & Watling, 2021; McGrath, Liljedahl, &
Palmgren, 2020) we have generated this sample by identifying the highest ranked scientific journals listed as covering the sub-topic

Table 1
Overview of included papers. If no location is indicated in paper, the Location column shows the location of the institution of first author.
Authors (year) Location Research question (paraphrased) Research methods

Attali and van der Kleij (2017)
Chen, Breslow, and DeBoer
(2018)
G. Cheng (2017)
M.-T. Cheng, Rosenheck, Lin,
and Klopfer (2017)
Clark-Gordon, Bowman,
Hadden, and Frisby (2019)
Filius et al. (2018)
Förster et al. (2018)
Lefevre and Cox (2017)
Lin (2018)
Luaces, Díez, and Bahamonde
(2018)
Nadolski and Hummel (2017)
Pardo et al. (2017)
Rasi and Vuojärvi (2018)
Thai, De Wever, and Valcke
(2017)
R. A. Thomas, West, and Borup
(2017)
Truskowski and VanderMolen
(2017)
Zaini (2018)
USA How does type and timing of automated quiz feedback affect Quantitative (trace data)
performance?
USA How do students interact with immediate automated Quantitative (trace data)
feedback, and how does it affect learning?
Hong Kong How does automated feedback influence quality of student Mixed methods (survey, reflective journal,
writing? interviews)
USA How can gameplay data be analysed to yield results that feed Quantitative (trace data)
back into the learning ecosystem?
USA How do college instructors use and perceive digital written Mixed methods (survey)
feedback?
Netherlands How can dialogic peer feedback lead to deep learning? Mixed methods (questionnaire, interviews)
Germany How does automated feedback on optional quizzes affect Quantitative (trace data)
learning?
UK Do students prefer immediate or delayed feedback and why? Mixed methods (trace data, interviews)
Taiwan How does knowledge of the peer reviewer’s identity affect Quantitative (questionnaire, Facebook
feedback quality, perception of learning and fairness? messages)
Spain What is the quality of feedback and grades provided by new Quantitative (trace data)
peer grading system?
Netherlands Does retrospective cognitive feedback improve learning in Mixed methods (questionnaire, trace data,
serious games? interviews)
Australia How can learning analytics be used to provide formative Quantitative (questionnaire, exam results)
feedback in large classes?
Finland How do students experience usefulness and emotional support Quantitative (questionnaire, test results,
of audio feedback? transcriptions of audio feedback)
Vietnam How does the flipped classroom approach influence self- Quantitative (Questionnaire and test
efficacy, motivation and perceived flexibility? results)
USA How does the frequency of social presence indicators vary Quantitative (content of video and written
between feedback formats? feedback)
USA How does annotated video feedback influence psychomotor Mixed methods (questionnaire)
skills and transfer?
Australia How does automated feedback influence identity Qualitative (interviews, think aloud
representations of learners? sessions)

3
L.X. Jensen et al. Computers & Education 173 (2021) 104271

‘e-learning’ in the Scimago online database. In 2019, only four educational research journals in this category scored above 1.5 on the
SCImago Journal Rank (SJR) indicator: Computers and Education (C&E); British Journal of Educational Technology (BJET); Journal of
Computer Assisted Learning (JCAL); and The Internet and Higher Education (I&HE). The SRJ indicator is a measure of a scientific journal’s
impact, influence and prestige (González-Pereira, Guerrero-Bote, & Moya-Anegón, 2010). We focussed on high-ranking journals
because we are interested in the understandings of feedback found in research papers that represent state-of-the-art research.
As with other critical reviews (e.g. Ashwin, 2012), we focus on a shorter period than what is usual in other types of reviews. To get a
sample representing the most current research the search was limited to papers published between January 2017 and February 2019,
which included the word “feedback” in the title. The commencement date (January 1, 2017) was chosen because that yielded enough
material for us to start seeing the same metaphorical language repeated across papers. By including only publications since 2017, we
were also able to keep our focus on what metaphors are currently used by researchers of feedback. This search returned 12 papers in
C&E, 7 in BJET, 4 in JCAL and 2 in I&HE, so altogether 25 papers. Of those 25, eight were discarded because they did not deal with
instructional feedback.
Although this process of search terms and selection criteria resembles that used in systematic reviews, it is important to note that this
study does not review studies and their findings, but forms a critical examination of authors’ language, in line with the critical literature
review tradition. Our focus is exclusively on the language of the included publications, and what that language implies about the key
phenomenon being discussed.
Table 1 presents an overview of the 17 included papers.

2.3. Identifying metaphorical language

To create our dataset of metaphorical language about feedback in online and digital learning we employed a range of linguistic
techniques (Cameron, 2003; Group Pragglejaz, 2007). First, we examined each direct and indirect reference to feedback in our selected
papers to determine if it was metaphorical or not. A direct reference would be the word feedback; an indirect reference might be
pronouns or synonyms for feedback such as comment, message, or information.
Some language excerpts were clearly metaphorical, while others simply referred to feedback without indicating any metaphorical
understanding. As an example, compare “feedback can be thought of as a cycle of information” to “students have mixed perceptions of
this form of feedback” (both from Truskowski & VanderMolen, 2017). Only the first excerpt would be noted down as metaphorical
language. However, authentic language examples may contain few explicit metaphors in the form feedback is a [some noun], but still
have rich metaphorical language that can be identified through metaphorical use of verbs, adjectives and prepositions. In our dataset
we included both expressions that explicitly state the metaphor, e.g. “feedback is a tool”, as well as less explicit metaphorical language,
e.g. if feedback “forces” students to do something, or students are “welcoming” the feedback. Often the implicit metaphorical language
is seen in the verbs: for example, learners can use, adhere to, or interact with feedback. Or the adjectives describing the feedback, such as
efficient, helpful, or addictive. Such implicit language was included in the dataset.

2.4. Identifying conceptual metaphors

There is always a tension between two considerations when identifying conceptual metaphors in real language excerpts. On the one
hand, a more specific metaphor has more entailments and offers a more detailed picture of our understanding of the phenomenon. On
the other hand, a too specific metaphor would maybe only apply to a single language excerpt, and thus cannot reasonably be said to
reveal much about our understandings. What we were looking for in the analysis were dominant conceptual metaphors. By dominant,
we mean conceptual metaphors that can be seen across many examples of metaphorical language, and across different papers.
The process of identification of conceptual metaphors was first done for each set of metaphorical language associated with a single
paper similar to open coding of qualitative data. We started out with a large number of codes and worked towards grouping them into
fewer and more robust themes, only in our case the themes were the proposed conceptual metaphors. This iterative process involved
multiple readings of metaphorical language in order to highlight, merge, reject and reorder the proposed conceptual metaphors. As an
example, we merged the understanding of feedback as encouragement and praise with the understanding of feedback as a basis for
student motivation. Later this was merged with the understanding of feedback as a generator of learner autonomy and empowerment
into an overall conceptualization of feedback as coaching.
Such merging of two proposed conceptual metaphors was done when they were strongly related with overlapping entailments,
especially in relation to how it envisions the feedback to have impact on learning. Another example of overlapping metaphors would be
FEEDBACK IS A DRUG and FEEDBACK IS A TREATMENT. Here the overlap is both in entailments (instructor exposes learner to effects
of feedback) and the phenomenon the metaphor is based on (a drug can be a treatment), and we opted to name the merged metaphor
after the more general of the two.
To enhance trustworthiness of the process, the coding of the metaphorical language was done independently by two researchers
(Author 1, Author 2) with similarities and differences discussed. Disagreements were seen as productive events that could be resolved
through discussion, in a few key cases involving Author 3.
To differentiate the metaphorical language (our data) from the conceptual metaphors (our findings), we will present the meta­
phorical language excerpts in quotation marks and the conceptual metaphors in ALL CAPS.

4
L.X. Jensen et al. Computers & Education 173 (2021) 104271

3. Results

Within each paper, we saw a variety of metaphors used for understanding feedback in online learning. Our analysis produced six
dominant conceptual metaphors. Each of them is associated with a certain understanding of what online learning is, and what role
feedback plays in the learning process. Consequently, each metaphor prescribes a certain online feedback practice, including a certain
division of responsibility and labour between the learner and other actors in the feedback process: instructors, peers, and instructional
technologies.
The conceptual metaphors are presented in Table 2, along with their primary entailments.
Each of the conceptual metaphors are described with further detail and quotes from the dataset below. The order of presentation
follows the locus of activity, going from teacher/system towards student-focussed metaphors.
Table 3 shows where the metaphors appear in the 17 papers, together with the feedback source and technological context. Most of
the papers included several dominant metaphors. The combination of metaphors does not follow a certain pattern. In fact each
metaphor can co-appear with any other metaphor. In the discussion section below, we explore what this may mean.

3.1. Feedback is a treatment

FEEDBACK IS A TREATMENT is one of the most noticeable conceptual metaphors in our data, and was identified in eleven of the 17
papers. According to this metaphor, feedback is the treatment or intervention that a learner (study participant) is exposed to, and the
outcome (learning) is then considered to be caused by the exposure to feedback. “Annotated video feedback appears to significantly
improve student learning” (Truskowski & VanderMolen, 2017) and “immediate and specific feedback by the course instructor can have
a dramatic effect on the quality of student learning” (Truskowski & VanderMolen, 2017) are both typical examples of metaphorical
language that sees feedback as a treatment. Papers that use FEEDBACK IS A TREATMENT are often inspired by the understanding of
feedback as an educational intervention with a large effect size as presented by e.g. Hattie and Timperley (2007). This type of
metaphorical language was common in our data, for instance Thai et al. (2017) write that “learning conditions in which feedback is

Table 2
Dominant conceptual metaphors. Characteristics of the feedback, the source of feedback (instructor, peer, system), the learner, and learning for
each of the dominant conceptual metaphors.
FEEDBACK IS A TREATMENT

Language example “immediate feedback with additional delayed review resulted in higher performance” (Attali & van der Kleij, 2017)
Nature of feedback A treatment with an effect size. Its efficiency is dependent on variables such as frequency, amount, content
How it influences learning Learning happens automatically in a learner who is exposed to feedback
Source of feedback A doctor who decides, prescribes and delivers the feedback treatment
Role of learner A patient, no agency beyond compliance. Risk of addiction or dependency
FEEDBACK IS A COSTLY COMMODITY

Language example
Nature of feedback
How it influences learning
Source of feedback
Role of learner
FEEDBACK IS COACHING
“instructors […] are under increasing pressure to dedicate resources and time for providing feedback” (Pardo et al., 2017)
An essential, but expensive, input into the learning process. Automation can help cut costs
Learning suffers when there is not enough feedback
A producer/provider of the resource
A consumer of the resource

Language example
Nature of feedback
How it influences learning
Source of feedback
Role of learner
FEEDBACK IS A COMMAND
“formative feedback can foster improved achievement and enhanced motivation to learn” (Chen et al., 2018)
Personal developmental attention aimed at motivating learners
Learning is effortful and requires motivation. Feedback can motivate the learner
A coach giving emotional and motivational support
A performer, not acquiring knowledge but improving performance

Language example
Nature of feedback
How it influences learning
Source of feedback
Role of learner
FEEDBACK IS A LEARNER TOOL
“they experienced pressures, control and power from automatic feedback” (Zaini, 2018)
Confrontational, seeks to exercise power over the learner
Negative relation between feedback and learning because of lacking learner agency
A commander. Often a system providing automatically generated feedback
A subordinate. Can resist the commands, or give in by adhering to them

Language example
Nature of feedback
How it influences learning
Source of feedback
Role of learner
FEEDBACK IS A DIALOGUE
“We can identify how students are using immediate feedback in solving online homework problems” (Chen et al., 2018)
A tool for learning, not for teaching. It only has an impact if used by the learner
Learning is complicated but can be made easier if useful feedback is at hand
The provider of the tool
An active agent using the tool, and the primarily responsible agent in the learning process

Language example
Nature of feedback
How it influences learning
Source of feedback
Role of learner
“We also queried if students would like to respond to the feedback through an audio file” (Rasi & Vuojärvi, 2018)
A conversation about student work. A linguistic, social process
Learning is participation, knowledge is co-constructed in social interactions
A participant in the dialogue. Often a peeror an instructor
Co-responsible for the direction and content of the dialogue

5
L.X. Jensen et al. Computers & Education 173 (2021) 104271

Table 3
Metaphors in each paper. Main metaphors appear numerous times within a paper while other metaphors appear just once or twice. The Feedback
source column lists the source of the feedback information that was the primary focus of the paper.
Authors (year) Main metaphor(s) Other metaphor(s) Feedback source Technological context

Attali and van der Kleij (2017) Treatment Command Automated Online quiz
Chen et al. (2018) Learner tool Costly commodity Automated Checkable answer feature
Coaching
G. Cheng (2017) Treatment Costly commodity Automated Writing evaluation system
Learner tool
Coaching
M.-T. Cheng et al. (2017) Learner tool Automated Educational game
Clark-Gordon et al. (2019) Treatment Dialogue Instructor Comments and track-changes in digital document
Filius et al. (2018) Dialogue Peer Peer review system
Förster et al. (2018) Treatment Coaching Automated Online quiz
Lefevre and Cox (2017) Treatment Automated Online quiz
Lin (2018) Coaching Treatment Command Peer Peer assessment system built inside Facebook
Luaces et al. (2018) Costly commodity Peer Moodle LMS peer review feature Workshop
Nadolski and Hummel (2017) Treatment Command Automated Educational game
Costly commodity Learner tool
Pardo et al. (2017) Costly commodity Dialogue Automated Learning analytics
Treatment
Rasi and Vuojärvi (2018) Coaching Command Instructor Audio feedback for mobile learning
Learner tool Dialogue
Thai et al. (2017) Treatment Instructor Guiding questions asked in LMS and in classroom
R. A. Thomas et al. (2017) Coaching Learner tool Instructor Video and written feedback in Canvas LMS
Dialogue
Truskowski and VanderMolen (2017) Coaching Dialogue Instructor Video annotation technology
Learner tool
Treatment
Zaini (2018) Command Treatment Automated Grammar and spell checker software

given in a timely manner will result in higher learning performance” and Förster et al. (2018) explicitly mention the “large variability
in effect sizes of feedback on performance”.
As with pharmaceuticals, where the need for patient behaviour (beyond compliance) is limited, this metaphor ignores what the
student does with the feedback information and considers progress in learning as a cause-effect function of the feedback message as
well as the “amount, frequency, and duration” (Clark-Gordon et al., 2019) of the feedback treatment. For instance, Lefevre and Cox
(2017) write that “immediate feedback leads to faster immediate acquisition but subjects who received delayed feedback achieved
higher grades in subsequent tests”, while Nadolski and Hummel (2017) argue that “feedback should be provided frequently and
timely” because “[t]he closer feedback is to actual performance (just-in-time), the more powerful its impact will be on subsequent
performance and learner motivation”. Lin (2018) concludes that “[f]eedback’s positive effect on learning appears to rely heavily on
comment content” and Pardo et al. (2017) suggest that future research should investigate “the ideal message wording to maximise
engagement”.
Originating in medical sciences, the research design that inspired this metaphor has been adopted by social and educational sci­
ences, who have retained the word treatment, even when no health-related outcomes are considered. Despite this, FEEDBACK IS A
TREATMENT is also seen in medicine-related metaphorical language in our data. Förster et al. (2018) write about providing feedback
“as a performance-enhancing measure to university students in large classes”, while also recognizing that it might “create a high level
of dependency in such a way that performance suffers when feedback is withheld”. Zaini (2018) describes how study participants
“admitted that they were dependent on the feedback given” and quotes one student for saying that she was “addicted” to automated
feedback she got from grammar- and spell-check software.
The most important entailment of this conceptual metaphor is that agency and responsibility lie with the instructor (or course
designer), who is choosing and providing the treatment. The learner just has to take the medicine. As illustrated by e.g. Nadolski and
Hummel (2017), Lefevre and Cox (2017), and Pardo et al. (2017), this is conducive to research that explores the content, frequency and
timing of feedback-messages, and feedback practices that focus on the optimal delivery and composition (rather than the learner’s use)
of feedback.

3.2. Feedback is a costly commodity

The conceptual metaphor FEEDBACK IS A COSTLY COMMODITY frames feedback as burdensome and expensive to produce/
procure, but at the same time as a necessary input into the learning process. It was identified in five of the 17 included papers. Pardo
et al. (2017) write about the “significant workload barriers for instructors that impede their capacity to provide timely and meaningful
feedback” even though they “are under increasing pressure to dedicate resources and time for providing feedback”. Similarly, G. Cheng
(2017) states that “feedback on writing is a time-consuming job and […] it can be problematic for teachers to provide feedback to
students on a regular basis”, and Chen et al. (2018) write that “in the context of higher education, it is often not possible for instructors
to provide timely feedback to every student individually.” Interestingly, the metaphor is not exclusive to instructor-generated

6
L.X. Jensen et al. Computers & Education 173 (2021) 104271

feedback. In a paper on peer-feedback Luaces et al. (2018) write that giving peer-feedback is a disadvantage for students because it
“increase [s] their already heavy workload”.
The focus of this conceptual metaphor is on the creation of feedback information, and consequently on the challenges connected
with being the generator of feedback, rather than the user of feedback. However it is expected that “students make use of feedback”
(Chen et al., 2018) after they receive it. This metaphor is often present in papers that argue for automation or scalability of feedback
provision in online education, as well as papers that explore how time-consuming are different ways of delivering and creating
feedback information.

3.3. Feedback is coaching

For FEEDBACK IS COACHING, the main purpose of feedback is to motivate students to action. This conceptualization understands
the feedback process as one where the instructor, like a coach, provides praise and encouragement to learners, in order to motivate and
empower them. Within this conceptualization, the effect that feedback has on learning is primarily through direction and enhancement
of student effort.
We identified this metaphor in seven of the 17 papers. In the words of Chen et al. (2018), “[t]imely and informative feedback can
help learners recognize and correct misconceptions, motivate them to acquire knowledge, and increase their confidence and moti­
vation to learn”. This means that feedback can have an effect on learning by “making students feel personally committed” (Filius et al.,
2018) which in turn can “motivate the learner to train longer” (Truskowski & VanderMolen, 2017). The metaphor understands online
learning as something that takes perseverance. However, in this understanding feedback does not directly prescribe student behaviour,
but is focussed on “foster [ing] student’s learning autonomy” (G. Cheng, 2017). Indeed, “feedback is one of the most powerful means of
empowering students, as well as enhancing and transforming their learning” (Rasi & Vuojärvi, 2018).
FEEDBACK IS COACHING highlights how learners react emotionally to feedback. An example is Rasi and Vuojärvi (2018) who
write that “[w]ell-timed, detailed, specific and positive feedback can empower students during their learning processes through
emotions such as joy, pride and excitement”, or G. Cheng (2017) noting that “teacher feedback could give [the online learners] a sense
of care and encouragement”.
As with FEEDBACK IS A TREATMENT and FEEDBACK IS A COSTLY COMMODITY, this conceptual metaphor considers feedback to
be the responsibility of the instructor, but it is connected to a hope or expectation of the online learner being motivated to continue
putting effort into learning.

3.4. Feedback is a command

The conceptual metaphor FEEDBACK IS A COMMAND describes feedback as controlling and directive. This metaphor, which
appeared in five of the 17 papers, is often seen in automated correctional feedback that, despite being well intentioned, may feel
intrusive, in part because it reduces the learner’s control over their own learning process. According to Rasi and Vuojärvi (2018) “[a]
udio feedback may be experienced as too confrontational by some students because of its ability to convey negative emotions” and
some learners “do not welcome audio feedback because of its intrusiveness” (Rasi & Vuojärvi, 2018).
This metaphor presents a negative understanding of feedback that researchers may find when they examine the experiences of
learners that are exposed to feedback. Zaini (2018) writes that “feedback can exercise power over and influence [language] learners’
identity representations” and describes how the learners “experienced pressures, control and power from automatic feedback”.
The corrective nature of FEEDBACK IS A COMMAND reduces the learner’s options for engaging with the feedback to either follow
the commands or reject to do so, with possible unpleasant consequences. One study participant told Zaini (2018) that “when I do not
follow the feedback, I am sure I am wrong and that I have to adhere to the comments” while another lamented that “she did not have
agency when confronted with feedback”.

3.5. Feedback is a learner tool

For FEEDBACK IS A LEARNER TOOL, agency lies with the learner, and any impact feedback might have on learning depends on it
being used by the learner. The metaphor was identified in seven of the included papers. According to this understanding, feedback can
empower students to manage their own learning process by engaging with and using feedback. Central to this metaphor is that the
learner “uses the feedback to learn” (Chen et al., 2018). Truskowski and VanderMolen (2017) write that feedback is “used to improve
performance during independent practice”, and M.-T. Cheng et al. (2017) describe how learners are “using the feedback from their
experiments to refine [their] strategy.” This focus on the student’s use of feedback as a part of their learning process can be seen as
contrasting with the focus on instructor’s use of feedback in the treatment and coaching metaphors.
The learning process is sometimes seen as a journey, in which case feedback is used as a tool for navigation, for instance it can be
“used to inform their next steps in a quest” (M.-T. Cheng et al., 2017) or “for learners to monitor their progress” (Nadolski & Hummel,
2017). Quality of feedback is measured in terms of its usefulness to the online learner, in fact “students expect quality feedback to be
useful” (Clark-Gordon et al., 2019), and generally the metaphor is relevant for research that examines what types of feedback are “easy
to use” (Rasi & Vuojärvi, 2018) or simply “perceived to be more useful” (R. A. Thomas et al., 2017). This means that feedback in­
formation is only linked to learning through the actions that students take after receiving it. Rasi and Vuojärvi (2018) measure
feedback effect “in terms of the revisions the students make to their assignments” and Chen et al. (2018) conclude that “certain patterns
of engagement with feedback reflect productive study strategies and significantly predict higher performance.”

7
L.X. Jensen et al. Computers & Education 173 (2021) 104271

3.6. Feedback is a dialogue

FEEDBACK IS A DIALOGUE was identified in six of the 17 included papers. With this metaphor, feedback is seen as a cyclical
process, “a continuing dialogue between the feedback provider and the feedback receiver” (Filius et al., 2018), where the participants
in the conversation share both agency and responsibility for creating a productive and meaningful feedback process. This metaphor is
the basis for the feedback practice referred to as dialogic feedback, which according to Filius et al. (2018) “emphasizes the importance of
interaction between the feedback provider and the feedback receiver beyond the initial feedback, as well as the importance of engaging
in a dialogue about the feedback”.
The FEEDBACK IS A DIALOGUE metaphor is often seen in the use of dialogue-related language – both the actions of learner and
instructor can be described as responding. Rasi and Vuojärvi (2018) describe how “marks were provided only after students had
responded to feedback comments” and R. A. Thomas et al. (2017) write that “[a]ll feedback was provided in Canvas as a response to
each assignment submission.” The dialogue related language is also seen when the feedback process is not directing the learner but
providing answers (Rasi & Vuojärvi, 2018). The dialogue metaphor highlights that feedback serves a social purpose and “plays an
important role in building instructor–student relationships” (R. A. Thomas et al., 2017), with certain forms of feedback feeling “more
interactive and intimate” (R. A. Thomas et al., 2017).

4. Discussion

As described above, there exists a variety of conceptual metaphors that frame our understandings of feedback in online learning.
[Redundant sentence deleted] The ways in which conceptual metaphors influence our thinking are subtle, and in everyday life, we
tend not to reflect on the entailments of certain conceptualizations. Because of this, the informal language used by online educators
may contain even more unreflective use of feedback metaphors, than can be found in the carefully worded academic papers from which
we generated our dataset. Consequently, the themes discussed below are not only relevant for online learning researchers but also for
online educators and instructional designers.
This section discusses the implications of our findings. First, we explore how the dominant conceptual metaphors and their en­
tailments relate to current ideas in more conceptual research on feedback. This leads us to a discussion of what the individual con­
ceptual metaphors can tell us about online learning researchers’ understandings of feedback and its role in online teaching and
learning. After this, we turn our attention to the fact that our analysis generated a variety of conceptual metaphors and discuss what
might contribute to this diversity in conceptualizations. Finally, we look ahead and discuss how online learning researchers and
practitioners may use the findings of this paper to reflect on their own conceptualizations of feedback.

4.1. Situating the metaphors in current feedback thinking

Because of the intimate connection between metaphor and practice, it is relevant to consider how the different conceptual met­
aphors we identified sit with what feedback researchers generally consider good feedback practices. One of the most critical differences
between the metaphors is how they envision the roles and responsibilities of different actors in the online feedback process. This is
frequently addressed in the conceptual literature on feedback and is generally considered to be at the heart of the challenge to improve
feedback practices. Researchers talk about a shift in feedback thinking, away from traditional ideas about feedback as something the
teacher gives to a learner, towards new conceptualizations that highlight the active role of the student in seeking, interpreting and
using feedback as part of their learning process (Boud & Molloy, 2013)(). Winstone and Carless (2019) refer to this as the old and new
paradigms. Central to these shifts in feedback thinking, is the idea that if students do not do anything with the information they receive,
then we should not assume that it has any impact on learning (Henderson, Ajjawi, Boud, & Molloy, 2019)().
In our dataset, we identified four teacher-centric understandings – FEEDBACK IS A TREATMENT, A COSTLY COMMODITY, A
COMMAND, AND COACHING – and two learner-centric understandings, namely FEEDBACK IS A LEARNER TOOL and FEEDBACK IS A
DIALOGUE. The fact that four of the metaphors align with views of feedback and learning that many feedback researchers consider
outdated may also reflect some more fundamental challenges facing online education. Although complex academic skills are being
taught online at universities across the world, there is perhaps a tendency for online learning to focus on learning outcomes in the
lower end of Bloom’s taxonomy, characterized by remembering and understanding facts (Bloom, 1956). Many online feedback
practices naturally tend towards being corrective or directional, and there might be less incentive to invest time in rethinking feedback.

4.1.1. The treatment metaphor

Since large scale meta analyses concluded that feedback has a powerful influence on student achievement (Hattie & Timperley,
2007), research on feedback has been permeated by the idea that feedback is a magic ingredient that educators can add to their online
courses in order to improve learning outcomes. In our data, this view of feedback’s role in online learning was most clearly exemplified
by FEEDBACK IS A TREATMENT. A possible issue with this understanding is that it may lure education researchers into recommending
unhelpful assessment designs in which feedback information is simply delivered without offering the student any opportunity or
reason to act on it. [Redundant sentences deleted].

4.1.2. The costly commodity metaphor

In online teaching, FEEDBACK IS A COSTLY COMMODITY is meaningful because individual feedback is often the most labour-
intensive element of otherwise scalable online courses. This metaphor may be useful to analyse expenses and divide workload, or

8
L.X. Jensen et al. Computers & Education 173 (2021) 104271

in studies that compare the cost associated with different feedback practices (e.g. Leger, Glass, Katsiampa, Liu, & Sirichand, 2017).
However, it has potential pitfalls because it does not clearly link feedback to any impact on learning. As shown by Bennett, Dawson,
Bearman, Molloy, & Boud (2017), educators who use technology to make provision of assessment feedback less time-consuming often
choose quick solutions, such as online quizzes, instead of more potentially fruitful alternatives. This illustrates how a feedback met­
aphor with no conceptualization of how feedback influences learning outcomes, may lead to feedback practices with little effect on
learning.

4.1.3. The coaching metaphor

It is well established that feedback can have a profound emotional effect on students (e.g. Rowe, Fitness, & Wood, 2014; Shields,
2015), especially those that are less emotionally mature (Pitt & Norton, 2017). In online learning, which is often characterized by very
limited personal contact between teaching staff and online students, this emotionally charged process might even be the main arena for
interaction between instructor and student. In such cases, FEEDBACK IS COACHING is useful because it sensitizes researchers and
online instructors to the negative effects that feedback (or lack thereof) may have on student motivation. However, using the feedback
process as the main vehicle for motivating online learners comes with possible drawbacks, because it makes motivation the primary
outcome of the interaction, and downplays the pedagogical purposes of feedback.

4.1.4. The command metaphor

When considering FEEDBACK IS A COMMAND is it important to remember that this metaphor is often used to describe coun­
terproductive feedback, which is overly corrective and reduces learner agency by excessively controlling behaviour. In our analysis, all
papers that use the command metaphor are investigating correct/incorrect evaluations of student interactions, such as spell- and
grammar-check software or online quizzes. Such information is arguably needed to evaluate micro-interactions in online environ­
ments, and automated correctional feedback is frequent in most online learning. The command metaphor may be appropriate for
feedback in what Spiro and DeSchryver (2009) call well-structured domains, but runs the danger of missing the important formative
functions of feedback when applied to the instruction of more complex domains of knowledge.

4.1.5. The learner tool metaphor

Conceptualizing feedback as a tool of the learner is associated with a view of online learning as an individual activity, and more
generally with constructivist ideas of knowledge and learning. One current area of inquiry that sits well with the learner tool metaphor
is the concept of feedback literacy, defined as “the understandings, capacities and dispositions needed to make sense of information and
use it to enhance work or learning strategies” (Carless & Boud, 2018)(). This work profits from a metaphor that highlights learner
agency and the learner’s capacity to use the tool. The metaphor’s focus on learning as an individual activity makes it less useful for
conceptualizing feedback within collaborative learning or peer-feedback frameworks that are often characterized by iterative feedback
processes and each participant’s dual role as creator and receiver of feedback information.

4.1.6. The dialogue metaphor

Unlike the solitary online learner of the learner tool metaphor, FEEDBACK IS DIALOGUE draws on theories of social learning and
what Sfard (2009) terms learning-as-participation. This metaphor is closely aligned with specific dialogic feedback practices (Nicol,
2010) and as such has stricter entailments related to the roles of learner, feedback source and instructional system that is the case for
FEEDBACK IS A LEARNER TOOL. Because of its entailments, the dialogue metaphor might only be appropriate for a certain subset of
online feedback practices that are designed around iteration and co-construction, such as peer-review systems (Filius et al., 2018) or
track-changes on written drafts (Clark-Gordon et al., 2019). The dialogue metaphor could be less appropriate in the context of online
courses characterized by self-study and automated feedback.

4.2. Multiple metaphors for one phenomenon

In a study investigating student engagement with feedback Price, Handley, Millar, and O’Donovan (2010) found that there was
little agreement among instructors and students as to what the purpose of feedback is. The existence of several dominant conceptual
metaphors for online feedback may illustrate a similar underlying disagreement among researchers about the role feedback should
play in online learning. However, it may be that the multiple metaphors are necessary, and indeed desirable, because the challenges we
meet when working with online feedback are legion. The research-worthy questions related to online feedback practices include such
diverse inquiries as how to keep down costs associated with providing feedback to growing number of students; how to use learning
analytics to improve outcomes; how to keep online students motivated; and how to design online courses that require students to use
the feedback information they receive in subsequent assignments. Maybe it is unlikely that one single metaphor is appropriate for
addressing a phenomenon that is stretched so wide.
Another reason for the presence of multiple dominant metaphors may be that the research on feedback in online education is not
always based on solid conceptual work, but rather uses it in an everyday sense to mean any information directed at an individual
learner. Consequently, conceptualizations of feedback in online learning changes with the purpose of the information and may lead to
confusion when we use the same word for what are essentially different practices.
The variation in conceptual metaphors is not just across papers. In most papers, we identified more than one dominant metaphor.
There may be good reason for operating with several metaphors. It is not necessarily inconsistent to consider online feedback to be both
a treatment and a costly commodity (e.g. Nadolski & Hummel, 2017). In other cases, however, the co-appearance of metaphors

9
L.X. Jensen et al. Computers & Education 173 (2021) 104271

happens when a learner-focussed metaphor is used in the introductory presentation of current research, while the rest of the paper uses
teacher-focussed metaphors (e.g. Pardo et al., 2017). This highlights the challenge of letting go of familiar and seemingly neutral
metaphors that consider the feedback process to be the responsibility of the instructor, even when the intention is to base the work on
learner-focussed conceptualizations. Papers with a single dominant metaphor do not necessarily have an advantage. The costly
commodity metaphor may, for example, be valuable in the rationale for a study, but insufficient when discussing the value of quiz
feedback. Moreover, there is no set algorithm for how multiple dominant algorithms may work together. While at surface metaphors
may appear in conflict, this may not be the case. Rather than conflict between metaphors, we see in the data that there may be a conflict
between the feedback metaphor employed and the feedback practice explored. As an example, Truskowski and VanderMolen (2017)
has both treatment and learner tool as main metaphors. On the surface, that is two of the most conflicting metaphors. However, in the
paper, the treatment metaphor is mostly used in the context of the quantitative analysis, and the learner tool metaphor is used when
discussing the qualitative part of the study. This diversity of metaphors makes sense with respect to the conceptualization of feedback
within the research design.
When discussing this diversity in how online learning researchers understand feedback, a relevant question is whether the different
metaphors describe different aspects of one phenomenon or perhaps several phenomena unhelpfully lumped together under one
heading. With the many new feedback practices that online education offers, the concept of feedback might have expanded so much
that it is difficult to discern from related processes such as instruction, self-monitoring, or help seeking. Of the 17 included papers, nine
investigated the use of automatically generated feedback – i.e. a format that is rarely found outside of online learning. Further con­
ceptual work is needed to explore if feedback in online learning research is best considered as a single phenomenon, and how it relates
to other processes and practices that learners and instructors participate in.

4.3. Implications

As illustrated, it is important that researchers are being conscious and deliberate in the language they invoke when writing about
feedback. This means reflecting critically upon conceptual metaphors, entailments and assumptions of their own conceptualizations.
Metaphorical understandings of feedback can be identified by asking questions about the roles and responsibilities of learners and
instructors in the feedback process. The answers to these questions hide in language use. Is feedback addictive or empowering? Should
the learner follow or use the feedback information? Is feedback a powerful treatment or a dialogic process? This paper encourages re­
searchers to ask such questions in order to employ a deliberate use of metaphoric language when conceptualizing feedback.
Furthermore, this deliberate use of metaphors can help researchers assess their suitability for the online teaching-learning practices
that are being investigated. Some feedback metaphors align better with certain feedback practices. Is costly commodity a suitable
metaphor for peer feedback practices, or would it sharpen the thinking to rely on e.g. a dialogue metaphor? Research that investigates
instructional feedback in language learning may get further with a feedback metaphor that highlights the learner’s reapplication of the
feedback comments in future work instead of the instructor’s production of the comments.
Beyond these implications for individual researchers, we hope that this work can also spark debate among scholars of digital
education. With the growing prominence of digital tools and media in education, and the blurring of the online-offline dichotomy, the
field has a lot to add to the general feedback research. Useful steps to further such an integration would be more conceptual work that
brings the notions of learning analytics, adaptive learning systems, video feedback, or serious games from the periphery to the
mainstream of feedback research.

4.4. Limitations

An important limitation to the findings is that sampling was restricted to papers from journals published in English. This means that
the metaphors discussed above only represent the understandings that are present in this subset of journals. Because our work is based
on an analysis of language, this limitation is substantial. Although the included papers did come from 10 different countries, it is
possible that a metaphor analysis of research papers published in other languages and cultures may result in a different set of con­
ceptual metaphors. Similarly, it is possible that expanding the search to more journals or those published before 2017, might have
yielded a different or larger set of dominant metaphors.
Another limitation comes from this being interpretative work. Other researchers might have opted for a higher number of more
specific metaphors, while others again might have preferred to identify just a few very general ones. This limitation is inherent in all
qualitative work, and findings should not be taken as a final list of all existing feedback metaphors, but rather an invitation to
interrogate our understandings of feedback and how they may influence research directions and recommendations.

5. Conclusions

This study analysed metaphorical language used by online learning researchers to identify ways that feedback is conceptualized in
the literature. The analysis generated six dominant conceptual metaphors, each representing a different conceptualization of what role
feedback plays in online learning. Some of these are likely also to exist in the feedback research that is focussed on classroom learning,
while others are more clearly linked with the focus on scalability, automation and micro-interactions that are popular in online ed­
ucation. Of these dominant metaphors, four align with feedback practices that are considered inappropriate among feedback re­
searchers, because they entail that the instructor is the main agent in the feedback process, and that the feedback provided to learners
automatically leads to learning. The exceptions are FEEDBACK IS DIALOGUE and FEEDBACK IS A LEARNER TOOL, which both align

10
L.X. Jensen et al. Computers & Education 173 (2021) 104271

well with what is increasingly accepted as good practice.

Among researchers and educators, the term feedback is used for very diverse practices and processes. The six suggested metaphors
offer a terminology that can help us unpack what we mean when we talk about feedback, and the analysis of their entailments may be
helpful for researchers to question how their own work is influenced by conceptualizations. To move the research on feedback in online
education forward, we propose that educators and researcher interrogate their own conceptualizations to make entailments explicit
and to ensure they align with their beliefs about feedback’s relation to learning. Deliberate use of metaphors to conceptualize online
feedback practices may be useful for this.

Author contributions statement

Lasse X Jensen: Conceptualization, Methodology, Formal Analysis, Investigation, Writing - Original Draft. Margaret Bearman:
Conceptualization, Validation, Formal Analysis, Writing - Review & Editing. David Boud: Conceptualization, Writing - Review &
Editing.

Acknowledgements

The authors would like to thank the anonymous peer-reviewers of this paper for their detailed, critical, and constructive comments.
No funding to declare.

References

Alvarez, I., Espasa, A., & Guasch, T. (2012). The value of feedback in improving collaborative writing assignments in an online learning environment. Studies in Higher
Education, 37(4), 387–400. https://doi.org/10.1080/03075079.2010.510182
Ashwin, P. (2012). How often are theories developed through empirical research into higher education? Studies in Higher Education, 37(8), 941–955. https://doi.org/
10.1080/03075079.2011.557426
Attali, Y., & van der Kleij, F. (2017). Effects of feedback elaboration and feedback timing during computer-based practice in mathematics problem solving. Computers
& Education, 110, 154–169. https://doi.org/10.1016/j.compedu.2017.03.012
Bälter, O., Enström, E., & Klingenberg, B. (2013). The effect of short formative diagnostic web quizzes with minimal feedback. Computers & Education, 60(1), 234–242.
https://doi.org/10.1016/j.compedu.2012.08.014
Bearman, M., Mahoney, P., Tai, J., Castanelli, D., & Watling, C. (2021). Invoking culture in medical education research: A critical review and metaphor analysis.
Medical Education. https://doi.org/10.1111/medu.14464
Bennett, S., Dawson, P., Bearman, M., Molloy, E., & Boud, D. (2017). How technology shapes assessment design: Findings from a study of university teachers. British
Journal of Educational Technology, 48(2), 672–682. https://doi.org/10.1111/bjet.12439
Bennett, S., & Oliver, M. (2011). Talking back to theory: The missed opportunities in learning technology research. Research in Learning Technology, 19(3).
Bloom, B. S. (1956). Taxonomy of educational objectives (Vol. 1, pp. 20–24). New York: McKay: Cognitive domain.
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712.
https://doi.org/10.1080/02602938.2012.691462
Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400–413. https://doi.org/10.1080/
02602938.2015.1018133
Cameron, L. (2003). Metaphor in educational discourse: A&C black.
Carless, D. (2015). Excellence in university assessment: Learning from award-winning practice. Routledge.
Carless, D., & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, 1315–1325.
https://doi.org/10.1080/02602938.2018.1463354
Chen, X., Breslow, L., & DeBoer, J. (2018). Analyzing productive learning behaviors for students using immediate corrective feedback in a blended learning
environment. Computers & Education, 117, 59–74. https://doi.org/10.1016/j.compedu.2017.09.013
Cheng, G. (2017). The impact of online automated feedback on students’ reflective journal writing in an EFL course. The Internet and Higher Education, 34, 18–27.
https://doi.org/10.1016/j.iheduc.2017.04.002
Cheng, M.-T., Rosenheck, L., Lin, C.-Y., & Klopfer, E. (2017). Analyzing gameplay data to inform feedback loops in the Radix Endeavor. Computers & Education, 111,
60–73. https://doi.org/10.1016/j.compedu.2017.03.015
Clark-Gordon, C. V., Bowman, N. D., Hadden, A. A., & Frisby, B. N. (2019). College instructors and the digital red pen: An exploratory study of factors influencing the
adoption and non-adoption of digital written feedback technologies. Computers & Education, 128, 414–426. https://doi.org/10.1016/j.compedu.2018.10.002
Evans, C. (2013). Making sense of assessment feedback in higher education. Review of Educational Research, 83(1), 70–120. https://doi.org/10.3102/
0034654312474350
Filius, R. M., de Kleijn, R. A. M., Uijl, S. G., Prins, F. J., van Rijen, H. V. M., & Grobbee, D. E. (2018). Strengthening dialogic peer feedback aiming for deep learning in
SPOCs. Computers & Education, 125, 86–100. https://doi.org/10.1016/j.compedu.2018.06.004
Förster, M., Weiser, C., & Maur, A. (2018). How feedback provided by voluntary electronic quizzes affects learning outcomes of university students in large classes.
Computers & Education, 121, 100–114. https://doi.org/10.1016/j.compedu.2018.02.012
González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of
Informetrics, 4(3), 379–391. https://doi.org/10.1016/j.joi.2010.03.002
Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2),
91–108. https://doi.org/10.1111/j.1471-1842.2009.00848.x
Group Pragglejaz. (2007). Mip: A method for identifying metaphorically used words in discourse. Metaphor and Symbol, 22(1), 1–39. https://doi.org/10.1080/
10926480709336752
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.
Henderson, M., Ajjawi, R., Boud, D., & Molloy, E. (2019). Why Focus on Feedback Impact?. The Impact of Feedback in Higher Education (pp. 3–14). Springer.
Holloway, I., & Todres, L. (2003). The status of method: Flexibility, consistency and coherence. Qualitative Research, 3(3), 345–357. https://doi.org/10.1177/
1468794103033004
Johnson, N. F. (2015). The work of theory in ed-tech research. Critical perspectives on technology and education. Springer.
Kluger, A. N., & DeNisi, A. (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention
theory. Psychological Bulletin, 119(2), 254.
Lakoff, G., & Johnson, M. (1999). Philosophy in the flesh (Vol. 4). New york: Basic books.
Lakoff, G., & Johnson, M. (1980). Metaphors we live by. Chicago, IL: University of.

11
L.X. Jensen et al. Computers & Education 173 (2021) 104271

Lefevre, D., & Cox, B. (2017). Delayed instructional feedback may be more effective, but is this contrary to learners’ preferences? British Journal of Educational
Technology, 48(6), 1357–1367. https://doi.org/10.1111/bjet.12495
Leger, L. A., Glass, K., Katsiampa, P., Liu, S., & Sirichand, K. (2017). What if best practice is too expensive? Feedback on oral presentations and efficient use of
resources. Assessment & Evaluation in Higher Education, 42(3), 329–346. https://doi.org/10.1080/02602938.2015.1109054
Lin, G.-Y. (2018). Anonymous versus identified peer assessment via a Facebook-based learning application: Effects on quality of peer feedback, perceived learning,
perceived fairness, and attitude toward the system. Computers & Education, 116, 81–92. https://doi.org/10.1016/j.compedu.2017.08.010
Luaces, O., Díez, J., & Bahamonde, A. (2018). A peer assessment method to provide feedback, consistent grading and reduce students’ burden in massive teaching
settings. Computers & Education, 126, 283–295. https://doi.org/10.1016/j.compedu.2018.07.016
Maier, U., Wolf, N., & Randler, C. (2016). Effects of a computer-assisted formative assessment intervention based on multiple-tier diagnostic items and different
feedback types. Computers & Education, 95, 85–98. https://doi.org/10.1016/j.compedu.2015.12.002
McGrath, C., Liljedahl, M., & Palmgren, P. J. (2020). You say it, we say it, but how do we use it? Communities of practice: A critical analysis. Medical Education, 54(3),
188–195. https://doi.org/10.1111/medu.14021
Mensink, P. J., & King, K. (2020). Student access of online feedback is modified by the availability of assessment marks, gender and academic performance. British
Journal of Educational Technology, 51(1), 10–22. https://doi.org/10.1111/bjet.12752
Nadolski, R. J., & Hummel, H. G. K. (2017). Retrospective cognitive feedback for progress monitoring in serious games. British Journal of Educational Technology, 48(6),
1368–1379. https://doi.org/10.1111/bjet.12503
Nicol, D. (2010). From monologue to dialogue: Improving written feedback processes in mass higher education. Assessment & Evaluation in Higher Education, 35(5),
501–517.
Norman, G., & Eva, K. W. (2010). Quantitative research methods in medical education. Understanding medical education. Oxford, UK: Wiley-Blackwell.
Nye, A., Foskey, R., & Edwards, H. (2014). Collegial reflection on the meaning of metaphors in learning: Emerging theory and practice. Studies in Continuing Education,
36(2), 132–146.
Pardo, A. (2017). A feedback model for data-rich learning experiences. Assessment & Evaluation in Higher Education, 1–11. https://doi.org/10.1080/
02602938.2017.1356905
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2017). Using learning analytics to scale the provision of personalised feedback. British Journal of
Educational Technology. https://doi.org/10.1111/bjet.12592
Pitt, E., & Norton, L. (2017). ‘Now that’s the feedback I want!’ Students’ reactions to feedback on graded work and what they do with it. Assessment & Evaluation in
Higher Education, 42(4), 499–516. https://doi.org/10.1080/02602938.2016.1142500
Price, M., Handley, K., Millar, J., & O’Donovan, B. (2010). Feedback : All that effort, but what is the effect? Assessment & Evaluation in Higher Education, 35(3),
277–289. https://doi.org/10.1080/02602930903541007
Rasi, P., & Vuojärvi, H. (2018). Toward personal and emotional connectivity in mobile higher education through asynchronous formative audio feedback. British
Journal of Educational Technology, 49(2), 292–304. https://doi.org/10.1111/bjet.12587
Rowe, A. D., Fitness, J., & Wood, L. N. (2014). The role and functionality of emotions in feedback at university: A qualitative study. Australian Educational Researcher,
41(3), 283–309. https://doi.org/10.1007/s13384-013-0135-7
Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional Science, 18(2), 119–144. https://doi.org/10.1007/bf00117714
Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Educational Researcher, 27(2), 4–13.
Sfard, A. (2009). Moving between discourses: From learning-as-acquisition to learning-as-participation. In Paper presented at the AIP Conference proceedings.
Shields, S. (2015). ‘My work is bleeding’: Exploring students’ emotional responses to first-year assignment feedback. Teaching in Higher Education, 20(6), 614–624.
https://doi.org/10.1080/13562517.2015.1052786
Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.
Spiro, R. J., & DeSchryver, M. (2009). Constructivism: When it’s the wrong idea and when it’s the only idea. In Constructivist instruction (pp. 118–136). Routledge.
Sutton, P. (2012). Conceptualizing feedback literacy: Knowing, being, and acting. Innovations in Education & Teaching International, 49(1), 31–40.
Tai, J., Canny, B., Haines, T., & Molloy, E. (2015). Building evaluative judgement through peer-assisted learning: Opportunities in clinical medical education.
Advances in Health Sciences Education, 21, 659–676.
Tai, J., & Sevenhuysen, S. (2018). The role of peers in developing evaluative judgement. Developing Evaluative Judgement in Higher Education: Assessment for Knowing
and Producing Quality Work, 156–165.
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context.
Computers in Human Behavior, 47, 157–167. https://doi.org/10.1016/j.chb.2014.05.038
Thai, N. T. T., De Wever, B., & Valcke, M. (2017). The impact of a flipped classroom design on learning performance in higher education: Looking for the best “blend”
of lectures and guiding questions with feedback. Computers & Education, 107, 113–126. https://doi.org/10.1016/j.compedu.2017.01.003
Thomas, L., & Beauchamp, C. (2011). Understanding new teachers’ professional identities through metaphor. Teaching and Teacher Education, 27(4), 762–769. https://
doi.org/10.1016/j.tate.2010.12.007
Thomas, R. A., West, R. E., & Borup, J. (2017). An analysis of instructor social presence in online text and asynchronous video feedback comments. The Internet and
Higher Education, 33, 61–73. https://doi.org/10.1016/j.iheduc.2017.01.003
Todd, Z., & Low, G. (2010). A selective survey of research practice in published studies using metaphor analysis. Metaphor analysis: Research practice in applied
linguistics, social sciences and the humanities, 26, 43.
Truskowski, S., & VanderMolen, J. (2017). Outcomes and perceptions of annotated video feedback following psychomotor skill laboratories. Journal of Computer
Assisted Learning, 33(2), 97–105. https://doi.org/10.1111/jcal.12167
Twining, P., Heller, R. S., Nussbaum, M., & Tsai, C.-C. (2017). Some guidance on conducting and reporting qualitative studies. Computers & Education, 106, A1–A9.
https://doi.org/10.1016/j.compedu.2016.12.002
Van der Kleij, F. M., Adie, L. E., & Cumming, J. J. (2019). A meta-review of the student role in feedback. International Journal of Educational Research, 98, 303–323.
https://doi.org/10.1016/j.ijer.2019.09.005
Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-
analysis. Review of Educational Research, 85(4), 475–511.
Winstone, N., Bourne, J., Medland, E., Niculescu, I., & Rees, R. (2020). “Check the grade, log out”: Students’ engagement with feedback in learning management
systems. Assessment & Evaluation in Higher Education, 1–13. https://doi.org/10.1080/02602938.2020.1787331
Winstone, N., & Carless, D. (2019). Designing effective feedback processes in higher education: A learning-focused approach. Routledge.
Zaini, A. (2018). Word processors as monarchs: Computer-generated feedback can exercise power over and influence EAL learners’ identity representations. Computers
& Education, 120, 112–126. https://doi.org/10.1016/j.compedu.2018.01.014

12