Palmer - Salway - Audio Description On The Thought-Action Continuum

Audio Description on the Thought-Action Continuum
Author(s): Alan Palmer and Andrew Salway

Source: Style, Vol. 49, No. 2 (2015), pp. 126-148
Published by: Penn State University Press
Stable URL: http://www.jstor.org/stable/10.5325/style.49.2.0126
Accessed: 15-02-2016 12:01 UTC
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/
info/about/policies/terms.jsp
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content
in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship.
For more information about JSTOR, please contact support@jstor.org.
Penn State University Press is collaborating with JSTOR to digitize, preserve and extend access to Style.
http://www.jstor.org
This content downloaded from 128.250.144.144 on Mon, 15 Feb 2016 12:01:58 UTC
All use subject to JSTOR Terms and Conditions
Audio Description on the Thought-
Action Continuum
Alan Palmer
Independent scholar
Andrew Salway
Uni Research Computing
Abstract: The term “audio description” refers to the spoken commentary that is
added to film soundtracks to enable visually impaired people to follow the action on the
screen by supplementing the information that is already available from character dia-
logue and other parts of the soundtrack. This essay introduces some core narratological
concepts and discusses how they are relevant to audio description research and practice,
especially to the issue of characters’ mental states. It then describes how corpus analysis
techniques can be used to analyze larger samples of audio description and develop a pre-
liminary classification of ways in which describers convey information about characters’
mental states. It closes by discussing the synergies between audiovisual translation, nar-
ratology, and corpus linguistics. The essay is intended to be a contribution to the debate
within the field of audio description studies regarding the possibility and desirability of
“objectivity” in audio descriptions.
1. I N TR OD U CT ION
The term audio description refers to the spoken commentary that is

added to film soundtracks to enable visually impaired people to follow
the action on the screen by supplementing the information that is already
available from character dialogue and other parts of the soundtrack. As
this essay explains, there is currently a debate within this field regard-
ing the possibility and desirability of “objectivity” in audio descriptions.
Although this debate may appear to be of limited interest to anyone who
is not directly concerned with the production and reception of audio
descriptions, we feel that it raises large and important issues that are
Style, Vol. 49, No. 2, 2015. Copyright © 2015 The Pennsylvania State University, University Park, PA
Style 49.2_02_Palmer_Salway.indd 126 22/05/15 12:41 AM

Alan Palmer and Andrew Salway 127
of relevance to all those who work in the much wider fields of narrative
theory, stylistics and literary theory generally. We have in mind, in par-
ticular, our questioning of the apparently obvious distinction between
descriptions of characters’ actions and descriptions of characters’ mental
processes in fictional narratives.
Specifically, this essay seeks to show how concepts from narratology and
techniques from corpus linguistics can be applied to the analysis of audio
description. We are concerned with the issue of what should be described in
audio description for feature films, and how it should be worded. We intro-
duce narratological concepts that can help to better articulate this issue and
to better analyze and compare examples of audio description. To comple-
ment the application of narratological concepts for a close reading of audio
description samples, we show how corpus linguistics techniques can be used
to learn about what is being described, and how, in a corpus of ninety-one
audio description scripts. Our focus is on how an audio description utter-
ance can, and, as it turns out, often does, describe a character’s actions as
depicted on-screen while simultaneously giving some information about the
character’s thoughts.
Central to the practice and theory of audio description is the question
of what should be included in a description (Vercauteren). Concerning the
issue of what to describe, and how to describe it, some recent guidelines for
audio description state:
“The best audio describers objectively recount the visual aspects of an image.
Subjective or qualitative judgments or comment get in the way – they constitute an
interpretation on the part of the describer and are unnecessary and unwanted . . .
Describers must differentiate between emotion or reasoning (which requires an
interpretation on the part of the observer) and the physical characteristics of emo-
tion or reasoning (which are more concrete and allow description users to conjure
their own interpretations).” (Snyder 17)
While striking an exceptionally unequivocal tone, these guidelines reflect a

point of view that has been prevalent among audio description practitioners.
Earlier guidance on standards for audio description provided by ITC is similar
in advising not to “interpret events” or “give away the plot” (ITC). The ITC guid-
ance also flags a contentious point about whether or not to give information

128 Style
“that is not apparent on the screen,” without really explaining how to determine
what is “apparent.” It seems to us that such guidelines leave unanswered ques-
tions about what information is necessary for an audience to understand and
enjoy a film, and what “interpretation” means in this context. Also, and this
becomes the main theme of the essay, with regards to what can be seen and
hence described, we take issue with the supposedly clear-cut distinction that
is made between a character’s “emotion or reasoning” (thoughts, which should
not be mentioned according to the guidelines) and their “apparent” physical
manifestations (actions, which may be described).
This essay seeks to contribute to audio description theory and practice in
two ways. First, we introduce more rigorously defined terms from narratol-
ogy to clarify and simplify the issues alluded to in the previous paragraph.
Second, we show how corpus linguistics techniques can be used to charac-
terize the information about characters’ actions and thoughts that audio
description actually provides. We, thus, hope to help circumvent the “mis-
guided dilemma of subjectivity versus objectivity” (Kruger and Orero 141)
and further develop the multidisciplinary nature of audiovisual translation
research (cf. Matamala and Orero).
Recent years have seen the uptake of various narratological concepts and
techniques by audio description researchers. Through a corpus-based analy-
sis, Salway (2007) showed how the language of audio description is influ-
enced by its narrative function, i.e., it must convey information about “chains
of events in cause-effect relationships” where “the agents of cause and effect
are characters with goals, beliefs and emotions” (152). With the aim of teach-
ing audio describers, Remael and Vercauteren (2007) show how to identify
“narratively significant cues” for inclusion in descriptions in relation to the
exposition phase of filmic narratives. Likewise, Ibañez (2012) argues for the
importance of explicating film narrative in the teaching of audio descrip-
tion. An innovative approach correlated eye-tracking data with viewers’ nar-
rative constructions to understand how different visual elements contribute
to storytelling (Kruger). Narratological theory was drawn on to argue for a
“more subjective and cultural-historical reference point for audio describers”
(Finbow 215) while Vandaele (2012) also extended consideration of narra-
tive beyond the depicted sequences of events in films to include the hypo-
thetical events that are triggered in the viewer’s mind.
Of particular relevance for the current essay is Kruger’s argument that
the filmic text, shorn of the moving image, must be “re-narrativised” for

blind and visually impaired audiences (Kruger 231–32). This means giving
consideration to how the story told by the film can be retold by com-
bining a new speech track with the dialogue, sound effects, and music.
Regarding the accessibility of films, Kruger makes the case for giving
less priority to a precise recounting of on-screen action and more pri-
ority to the “narrative implication or effect of what can be seen” (234).
To achieve this, “audio narration” (233) is suggested as an alternative to
audio description, where there is a continuum from objective description
(that may be achieved in the audio description of a documentary) to a
narration that may use, for example, “markers of focalization . . . to enable
the audience to situate themselves in the fictional world” (243). At the
core of the current essay is the somewhat analogous and complementary
thought–action continuum.
Informed by a narratological perspective, our premise is that understand-
ing a story entails knowing about the mental states of its characters. In other
words, it is essential for the audience of a film to learn about the causal net-
work of beliefs, desires, and goals that motivate and explain the characters’
actions. At first glance, this perspective may seem to be at odds with the
guidelines mentioned above: can it be possible to see, and hence to describe
mental states? Within the moving image of a film—for which audio descrip-
tion is a surrogate—actors act, and with their actions and through their
facial expressions, they reveal something about characters’ mental states;
of course, the other components of a film (dialogue, music, and film tech-
niques) contribute to the storytelling too.
The central question that we address here is this: how much information
about the “minds,” i.e., mental states, of characters in films is audio descrip-
tion able to convey—given the constraints of being objective (not “interpret-
ing”), allowing audiences to make their own inferences about a story (not
“giving away the plot”), and the limited time available for description (which
must fit between the film’s dialogue)?
In section 2, we introduce some core narratological concepts and discuss
how they are relevant to audio description research and practice, especially to
the issue of characters’ mental states. In section 3, we describe how corpus anal-
ysis techniques can be used to analyze larger samples of audio description and
develop a preliminary classification of ways in which describers convey infor-
mation about characters’ mental states. We close in section 4 by discussing the
synergies between audiovisual translation, narratology, and corpus linguistics.

130 Style
2. A
PPLY ING A THEO RY OF F ICTI O N AL M I N DS TO
AU D IO D ESCRIPTION
In section 2.1, we introduce some fundamental concepts from the field of

narratology and discuss how these can clarify thinking about audio descrip-
tion with an emphasis on the thought–action continuum. In section 2.2, this
is used to compare two audio descriptions of the same film in terms of how
they provide information about the minds of the characters.
2.1 Why Should Audiovisual Translation Be Interested in Narratology?
Narratology (or narrative theory) is the branch of literary study that is

devoted to the analysis of narrative. In its modern form, it grew out of the
French Structuralist movement of the 1960s. It has, however, recently become
more eclectic and inclusive in its use of the techniques used to analyze nar-
ratives (Herman). Narratology studies the nature, form, and functioning
of all narratives irrespective of their mode or medium of representation.
Indeed, many narratological concepts are particularly suited to multimodal
analysis (Ryan). For example, narratologists distinguish between story
(often referred to by the Russian term fabula) and discourse (sjuzet). Story
is the content plane, the “what” of narrative, the narrated; discourse is the
expression plane, the “how” of narrative, the narrating. So, the story of
Cinderella can be told differently in any number of different discourses or
media (a short story, a film, a play, a ballet, a cartoon, etc.) and still remain
the same story. The application of narratological concepts to a multimodal
activity such as audio description, which uses language to describe film,
should prove to be especially fruitful. For example, audio description tends
to be what narratologists call behaviorist narrative. Behaviorist narratives are
those that are limited to conveying characters’ words and actions and do not
give direct access to their thoughts and feelings. However, some narratolo-
gists emphasize how much information even behaviorist narratives are able
to convey about characters’ mental states (Palmer 206–7), and it is hoped
that the discussion that follows will contribute to this debate.
Within narratology, the study of what are called fictional minds is the
study of the mental functioning of the characters who inhabit the story-
worlds created by fictional narratives. For example, in relation to written
narratives, it addresses the question: how do we, when reading a novel, con-
struct from the words in the text an awareness of the mental functioning

of the characters in that novel? Readers enter the storyworlds created by

novels and then follow the logic of the events that occur in them primarily
by attempting to reconstruct the fictional minds of the characters in that sto-
ryworld. Otherwise, readers lose the plot. These constructions of the minds
of fictional characters by narrators and readers are central to our under-
standing of how novels work, because fictional narrative is, in essence, the
description of fictional mental functioning. It is not possible to follow the
plot of a novel without following the thought processes of the main charac-
ters in that novel. In fact, the plot consists to a great extent of those thought
processes.
As suggested by the reference above to the multimodal nature of narratol-
ogy, this approach can be applied as easily to films as to novels. So, to put the
issue of audio description in the context of this work in narrative theory, it
could be argued that the point of audio description is to help the audience to
follow the mental functioning of the characters in the film. For example, the
guidance on information and objectivity can be usefully restated in terms of
fictional minds. Descriptions should contain (among other things) essential
information about fictional minds but should avoid both obvious and unnec-
essarily subjective inferences about fictional minds.
It may appear at first glance that there is a tension here between this cog-
nitive focus on characters’ minds and the guidance that audio descriptions
should be as objective as possible. However, the purpose of the following
discussion is to show that there is no conflict. Even the most apparently
objective action descriptions often make important inferences about charac-
ters’ mental functioning in order to progress the plot.
It is tempting to think of action descriptions and descriptions of conscious-
ness as two very different things. The former describe physical movements, and
so on, and the latter convey private flows of thought. But it is not as simple
as that. There is a continuum rather than a simple dichotomy. It is called the
thought–action continuum (Palmer 212–14). Here is a simple example. “They
are hiding behind the curtain” looks like an action description pure and simple.
But compare it to another, similar phrase: “They are standing behind the cur-
tain.” In the context of the second example, the first description, “hiding,” starts
to look very different. It contains important information about the mental func-
tioning of the people standing behind the curtain because it explains the reason
why they are doing so. Saying they are “standing” there leaves open any num-
ber of reasons why they would be standing there. From this angle, the more a

132 Style
reader looks at the word “ hiding,” the more like a description of consciousness
it becomes. Put another way, the word “standing” is at the action end of the
thought–action continuum; the word “hiding” is nearer the middle, because
it describes the action but also contains a reference to the mental functioning
behind the action. It is worth mentioning that the mental states referred to in
words in the middle of the continuum are often, unsurprisingly of course, rea-
sons, motives, and intentions relating to action.
In the Philosophical Investigations (1958), the philosopher Ludwig
Wittgenstein quotes the statement “I noticed that he was out of humour,”
and asks, “Is this a report about his behaviour or his state of mind.” When
“Wittgenstein’s question” (see Palmer 120–21) is asked of the statement
“They are standing behind the curtain,” the answer is that it is a report of
their behavior; when asked of the statement “They are hiding behind the
curtain,” the answer is that it is a report of both their behavior and their state
of mind. And Wittgenstein’s question can usefully be asked of many of the
statements made in action descriptions.
The other large group of mental states that often occur in these statements
is emotions. For example, in one of the scenes from the film The English
Patient to be discussed in the following section, one of the audio descrip-
tions refers to the female protagonist, Katherine, as simply stopping and
turning toward Almasy. That is at the action end of the continuum. The other
description says that “She rounds on Almasy.” The second description con-
veys much more information about her emotions of anger and frustration.
So it is in the middle of the continuum. Similarly, for example, in the same
scene, “Katherine glares” at Almasy, rather than the more neutral “Katherine
looks at him.” A hypothetical example at the thought end of the continuum
would be something like “Katherine is angry with Almasy.”
2.2 Analyzing Three Scenes from The English Patient
The broad framework outlined above can now be applied to a comparison

between two different audio descriptions (shown as “A” and “B” below) of
three short scenes from Anthony Minghella’s film, The English Patient (1996).
All of these scenes occur fairly early in the film and show the beginnings
of the love affair between the two main characters, Katherine and Almasy,
together with the growing concern of Katherine’s husband, Geoffrey Clifton.
The discussion will look at both the similarities and the differences between
the two audio descriptions of these three scenes.

Scene one: As part of the communal entertainments around the desert campfire,
Katherine is telling the expedition party a story:
“Katherine hesitates as she meets Almasy’s gaze across the campfire.” (A)
“Katherine catches Almasy’s eye. He is transfixed. Clifton’s smile fades. Katherine

smiles awkwardly.” (B)
B is more informative than A, even though it is only four words longer,

telling us in only thirteen words a good deal about the states of mind of the
three characters. With regard to A, we are given no clue as to the various,
very different reasons why Katherine might hesitate. With regard to B, it is
a rewarding exercise to ask Wittgenstein’s question here. Is Almasy looking
transfixed (an action) or feeling transfixed (a state of consciousness) or both?
It is suggested that in the case of all of the four sentences in B, the answer
to the question is: both. These are reports of characters’ behavior and also
of their states of mind. In other words, all of the four sentences would be
placed in the middle of the thought–action continuum. The audience will
know that Katherine and Almasy are very awkward with each other because
their relationship is growing. Each is intensely aware of the other. Equally,
the description of Clifton’s behavior, his smile fading, clearly indicates his
state of mind. He is uneasily aware of the growing attraction between his
wife and another man. However, as could be expected, the description also
leaves open some intriguing areas of speculation and doubt. For example,
saying that Katherine catches Almasy’s eye raises issues of motives and
intention. Does she intend to catch his eye, or does she do it accidentally?
It is likely that listeners to the audio description will disagree on how they
interpret Katherine’s behavior within the context of the development of the
relationship so far in the film.
Scene two: During a dance at the Cairo hotel, Katherine has just finished dancing with a
member of the group:
“Almasy steps between them and takes Katherine in his arms. Seeing them, Geoffrey
sips his wine thoughtfully. On the dance floor, Almasy does not answer. He holds
her stiffly in his arms, gazing directly at her finely sculpted features and pale blond
hair. Disconcerted, Katherine meets his gaze, then glances away uncertainly. Without
speaking they glide across the floor between the dancing couples circling beneath
the crystal chandeliers.” (A)

134 Style
“Almasy cuts in and leads Katherine. She smiles. Almasy studies her intently and
she drops her gaze. When she lifts her eyes again he is still staring. He holds her in
a stiff embrace, his face inches from hers, his hand resting on the small of her back
pressing her close to him.” (B)
In this scene, the two descriptions are quite similar in some ways: both
emphasize Almasy’s taking control of the situation; both use the word “stiff ” to
describe Almasy’s dancing; and both have Katherine glancing away. However,
there are some differences. Version A makes more of Katherine’s turning
away movement, using the words “disconcerted” and “uncertainly” to provide
important information about her mental states. Also, it is odd that, as in the
last scene, only one description takes note of Clifton’s reaction to the relation-
ship between Katherine and Almasy. But this time, it is A rather than B.
In addition, the difference in what may be called focus is interesting.
B emphasizes the almost claustrophobic intimacy of the physical contact
between the pair: his face inches from hers, pressing her close. It con-
veys the tension and the growing attraction between them. By contrast,
A stresses the social and physical context of the dancing couples and the
crystal chandeliers. The use of the word “glide” suggests a much more
relaxed encounter. Very different mental states are implied: it makes the
two characters’ states of mind sound misleadingly relaxed and uninvolved
by comparison.
Scene three: Back in the desert, Katherine shows Almasy some of her paintings:
“She offers them to him. He takes the paintings and looks at them. He holds them
out. She takes the pictures back. Katherine walks a few steps then stops and turns. He
looks embarrassed. Quickly she walks away. Almasy looks after her uncertainly.” (A)
“Almasy takes them and glances up at her. Katherine smiles as he looks through
them. He hands them back. Katherine’s face falls. Reluctantly she takes the paint-
ings and starts to go. Then hesitates. She rounds on Almasy. He opens his mouth to
speak, but says nothing. Katherine glares, turns and marches off, heading up the side
of the dune. Almasy blinks, sighs and shakes his head. He returns to his book.” (B)
In this scene, B is noticeably more involved in Katherine’s emotions

than A. In narratological terms, the scene is focalized through her. It is pre-
sented from her point of view. The words used are more vivid: she “smiles,” she
“hesitates,” she takes the paintings “reluctantly.” She “rounds” on Almasy, she

“glares,” “turns and marches off.” Much of this is veering toward the thought
end of the thought–action continuum. Her conflicted thought processes are
made clear. By contrast, A stays at the action end of the continuum in rela-
tion to Katherine. Significantly though, the A focus is on Almasy’s mental
functioning: he “looks embarrassed”; he looks after her “uncertainly.”
We did not notice any consistent patterns of difference between the two
descriptions across the three scenes. For example, Clifton’s reactions were
noted in description B for scene 1 and in description A in scene 2. So there
were no obvious patterns of identification, say, with one character in one
description and another character in another. It may be that this point could
form the basis of at least one good criterion for objectivity, i.e., descriptions
should not over-identify with, or get over-involved in, one character at the
expense of the others. What we did notice in both descriptions were some
common devices used to give information about mental states. These include
descriptions of different types of smile (“smile fades,” “smiles awkwardly”)
and different ways of looking at other characters (“studies her intently,”
“looks after her uncertainly,” and descriptions of characters appearing to be
in a particular state (“he is transfixed,” “disconcerted,” “looks embarrassed”).
These devices become the starting point for our corpus analysis in section 3.
Given that the two descriptions are so informative about the three main
characters’ states of mind, the question arises as to whether they meet the
guidelines’ requirement to be objective. Our close reading of some audio
description fragments leads us to claim that it is possible to remain objec-
tive while being informative about mental states in a concise and consistent
way, and that such information is indeed essential for the audience to under-
stand the story. To put the point in narratological terms: the descriptions
are in appropriate places on the thought–action continuum. We have also
shown how small differences in descriptions of characters’ actions can result
in big potential differences for an audience’s understanding of a story, which
is perhaps something to be considered in future guidelines.
3. A
PPLY ING CORP U S A NALY S I S TECH N I Q UES TO
AUD IO D ESCRIPTION
In this section, we seek to demonstrate the utility of corpus analysis

techniques for surveying common words and phrases, and hence, the com-
mon kinds of information conveyed by audio description. In particular,
we will use these techniques to gauge the extent to which our samples of

136 Style
audio description provides information about characters’ mental states,

and to develop an account of the options audio describers have for convey-
ing such information. Automated text and corpus analysis techniques are
used in many fields, for many purposes, and a review of these is beyond the
scope of this chapter—for an introduction see Biber, Conrad, and Reppen
(1998). In broad terms though, individual texts can be compared accord-
ing to the frequencies of words and phrases occurring in them, and the
co-texts in which these words are used. In section 3.1, we use the “Word
List” and “Concordance” functionalities of the freely available corpus analy-
sis tool AntConc (Anthony) to extend the previous comparison of two audio
description scripts. In section 3.2, we apply these techniques, and others, to
a corpus comprising British English audio description scripts for ninety-one
films.
3.1 Further Comparison of Descriptions A and B
The detailed analysis that follows may be summarized in this way. Some
of the ways in which audio describers can give information about charac-
ters’ mental states are listed here roughly in order from action-oriented to
thought-oriented. This order implies an increase in the extent to which the
describer makes explicit for the AD audience what the sighted audience is
expected to infer from the moving image.
(1) Description of simple actions that are intended to imply mental states,
e.g., “Robin rolls his eyes.”
(2) Description of facial expressions, such as “smiles,” “frowns,” and “grins.”
(3) Modification of the description of actions by adding an adverb, e.g.,
“walks cautiously,” by using a troponym, e.g., “creeps,” or with a phrase
like “smiling in relief.” The particular choice of adverb, troponym, or
abstract noun will lead to a different positioning on the thought–
action continuum.
(4) Description of a character as appearing to be in a particular mental
state, e.g., “Harry looks confused.”
This is, at best, a preliminary attempt at a categorization. It does not

include actions such as “hide” (f=60), most likely because they do not take
a distinctive form or co-text in audio description. This suggests that our

ata-driven approach may need to be complemented by an approach that

d
starts with a semantic classification of verbs listing verbs such as “hide” that
fall midway on the thought–action continuum. However, what is certain at
this stage is that British audio description is rich in descriptions of charac-
ters’ minds—including information about their motivations, emotions, and
feelings toward others—that are described in a relatively constrained vari-
ety of forms, and that there seems to a be a tendency toward descriptions
around the midway point of the thought–action continuum (summarized
as point 3 above). What we tend not to see examples of are “pure thought”
descriptions, such as “Harry feels confused.”
Let us now look at a detailed comparison of two audio description
scripts for three scenes of one film (section 2.2). Certain words stood out
for the ways in which they informed about characters’ mental states: the
word smiles, especially when followed by an adverb; the word looks fol-
lowed by an adjective or an adverb; and, words describing ways of looking
such as gazes and glances. How are these and other words used through-
out the two descriptions to convey information about characters’ minds?
A frequency count showed that in the whole of description A, the word
form “smiles” occurred forty times, and in B it occurred fourteen times.1 In
eight instances in A, “smiles” was followed by an adverb that seems to give
more insight into a character’s mental state than “smiles” alone; in B there
were two instances. All ten examples are listed below.
She smiles benignly.

She smiles fondly at the man with the melted face.
She smiles fondly at him.
Katherine smiles awkwardly.
She smiles fondly and looks over at Almasy.
He smiles briefly at her in the darkness.
She smiles weakly.
He smiles ruefully and lowers his head, looking pensive.
Almasy smiles faintly.
Lady Hammond smiles delightedly as Katharine crosses the courtyard.
It is unsurprising that the writers of audio descriptions rely on stock or

standard devices that enable them to convey information about characters’

138 Style
mental states. These frequently recurring terms may be thought of as “pegs”

on which intention-rich adverbs can be hung. It seems that the “smiles” peg
is congenial to A but less so to B. Other favorite pegs include “looks” and
“walks” and these are discussed below. It is easy to see why “smiles” and
“looks” should be popular pegs; after all, the face can be very expressive of
internal mental states. The examples listed above are all in the middle of the
thought–action continuum because they are descriptions of actions that con-
vey states of mind. Perhaps “briefly” and “faintly” are nearer the action end
of the spectrum than the others because they require a little more context
to make the indicated state of mind clear, e.g., musical cues or other filmic
devices.
The word “looks” occurs fifty-six times in description A and thirty-one
times in B. It is used in these audio descriptions in two main senses: to look
at someone or something, and to appear to be in a certain mental state. It
is used in the latter sense seven times in both descriptions A and B. The
use of “looks” in the latter sense (appearing to be in a particular mental
state) is close to the thought end of the spectrum. This is because there are
not really any actions being described. It is a much more direct attempt
than the former sense to convey mental states. “He looks sad” could easily
be reformulated as “he appears to be sad.” This latter formulation would
make it clear that there is no action involved. In the following examples of
the latter usage, the apparent state of mind is very clearly indicated.
Almasy looks thoughtful.

He looks uncertain.
Katherine is watching but looks troubled.
In the cockpit Clifton looks grimly determined.
The patient looks drained.
She looks happy for him.
Then Hana looks grave again.
Hana looks concerned.
Madox looks puzzled.
He looks puzzled.
Almasy looks annoyed.
He looks embarrassed.
He looks upset and agitated.
She looks unhappy and tearful.

The uses in the former sense (looking at something) tend to be nearer the
middle of the spectrum because there is an action involved. In both descrip-
tions A and B “looks” is modified by an adverb seven times to give more
insight into mental states. It is clear that, even in the action uses of “looks,”
there is plenty of information being given about mental states. Indeed, it
is notable that several words convey the presence of a good deal of highly
self-conscious thought: “thoughtfully,” “pensively,” “uncertainly,” “curiously,”
and “questioningly.” This is not surprising given the subject matter of this
film—highly intelligent and sensitive people falling in love.
He looks thoughtfully from the hand to the surrounding rocks.

. . . he looks thoughtfully in the direction Katherine has taken.
He looks pointedly at Katherine.
Hana looks after him forlornly.
He sets the syringe to one side and looks directly at the patient.
He looks about him furtively.
He looks gravely down at her, unsure.
She looks pensively at the bell-tower of the monastery.
She heaves a dusty mattress on-to the bed and looks around thoughtfully.
He looks at her awkwardly.
Almasy looks after her uncertainly.
. . . and looks at him curiously as he digs in the sand.
She looks at him questioningly.
Hana touches his arm and looks at him fondly.
Of course, aside from smiling, looking, and walking, there are many other
actions that could be modified by adverbs to inform about mental states in
audio description. In order to focus on how these actions are performed, the
next step was to search the two descriptions for words taking a common and
easily identifiable form of English adverbs, i.e., those ending in -ly. From
inspecting the concordances of -ly adverbs, we found that in description
A over 100 of the 149 instances convey information about characters’ mental
states; in B it is over 90 of 136 occurrences.
As an alternative to modifying “looks” with an adverb, audio describers
working in some languages, such as English, have a variety of troponyms at
their disposal with which they can specify a particular manner of looking,
and in so doing give more insight into characters’ mental states, e.g., “gazes”

140 Style
and “glares.” We sourced a list of troponyms of “looks” from the online

lexical database (Fellbaum) to see how frequently these words occurred in
descriptions A and B (see Table 1); their concordances were inspected to
check that they were being used in the sense of a troponym of “looks.”
A uses a total of forty-seven of these troponyms, whereas B uses only
fifteen. The choice of words in this case does not make much difference to
the place of a statement on the continuum. The main difference appears to
be the degree of attention (“stares” denotes more attention than “glances”)
but the word “stare” in itself does not convey the reason for the attention.
For this, the context is required. However, the choice of “glares” over the
basic “looks” does suggest a feeling of anger from one character toward
another. We also counted the frequencies of troponyms of “walk.” The
following, which may be indicative of mental states as well as physical
condition, were found at least once in one or both of A and B: “creeps,”
“hobbles,” “staggers,” “steps,” “strides,” “stumbles,” “swaggers,” “toddles,”
“totters,” “waddles.”
The data presented above allows us to say something about the extent to
which each description conveys information about characters’ minds: over-
all, description A contains many more of the words that we take to be related
to characters’ minds than description B. Our method used relatively simple
corpus analysis techniques to good effect, but it is important to note that
the need in places for manual inspection of concordances means that it will
Table 1: Frequencies of troponyms of “looks” in descriptions A and B.
Troponym A B
stares 16 4
gazes 9 5
peers 8 3
glances 5 2
eyes 4 0
studies 3 0
glares 2 0
regards 0 1

be impractical to replicate parts of this analysis on a much larger sample

of audio description. Another problematic issue to be aware of is that the
analysis was led by our prior assumptions about what words are used to con-
vey information about characters’ minds. While our results support the idea
that “smiles,” “looks,” their troponyms, and -ly adverbs are indeed important
devices for audio description,2 we must recognize that there may be other
important devices that we have not accounted for.
3.2 Information about Mental States in a Corpus of Audio

Description Scripts
It could be argued that the prevalence of information about characters’

minds observed above was due to the choice of film, which is a drama
intimately concerned with the mental states of its protagonists. To under-
stand more generally how audio description conveys information about
characters’ mental states, and to see how we can scale-up our analysis,
we now turn to a corpus of ninety-one British English audio description
scripts compiled to be representative of major film genres. For more details
about the corpus and a corpus-based investigation into audio description
as a “special language” see Salway (2007). The analysis here is based on the
distribution of word forms and the co-texts in which they occur. We start
by looking at -ly adverbs, at their usage with “looks” and “walks” and at
troponyms for these verbs. We then extend our search for words germane
to the current analysis in three ways: (i) by automatically generating lists
of words using co-textual cues such as “looks -ed” and “smiling with. . . ”;
(ii) by using WordNet as previously; and (iii) by drawing on results from an
earlier analysis of this corpus. For the most part, the analysis was carried
out using the previously mentioned “Word List” and “Concordance” func-
tionalities. However, to easily extract lists of words occurring in particular
contexts, some programs were written in the Perl programming language3
which is optimized for text processing with regular expressions.4 The over-
all impression that we get from the analysis reported below is that infor-
mation about characters’ minds is generally widespread in the sample of
audio description available to us.
In our audio description corpus, the top twenty -ly words, with their fre-
quencies, are “slowly” (f =439), “gently” (f=190), “quickly” (f=148), “nervously”
(f =125), “suddenly” (f =125), “carefully” (f =114), “thoughtfully” (f =78), “anx-
iously” (f =70), “intently” (f =69), “desperately” (f =55), “happily” (f =52),

142 Style
“sadly” (f =50), “awkwardly” (f =46), “frantically” (f =42), “cautiously” (f =40),

“tenderly” (f =40), “wearily” (f =37), “curiously” (f =36), “angrily” (f =32),
“warily” (f =32). These alone, in addition to a long tail of less frequent words,
total 1,874 instances which, make over twenty instances per film on aver-
age. Let us now consider how two commonly described actions—looking
and walking—are modified with -ly words (see Tables 2a and 2b). It seems
that each action has a preferred set of adverbs, most (if not all) of which add
information about mental states.
Table 2a: Frequencies of word forms ending -ly that follow the word forms “looks.”
directly 9
anxiously 8
sadly, steadily, thoughtfully 7
nervously 6
fearfully, grimly, longingly, quizzically 4
curiously, slowly 3
blankly, briefly, carefully, frantically, intently, pensively, searchingly, 2
seriously, sheepishly, sympathetically, uneasily, urgently
accusingly, appealingly, awkwardly, bitterly, calmly, cautiously, closely, 1
coldly, contemptuously, defiantly, disappointedly, disapprovingly,
earnestly, enquiringly, entreatingly, firmly, forlornly, gravely, guiltily,
impatiently, incredulously, malevolently, meaningfully, miserably,
mournfully, pathetically, pityingly, pleadingly, pointedly, questioningly,
quickly, resignedly, skeptically, shakily, sharply, shyly, solemnly, surrep-
titiously, tenderly, unsurely, vacantly, warily, wistfully
Table 2b: Frequencies of word forms ending -ly that follow the word form “walks.”
slowly 57
purposefully 9
briskly 8
carefully, cautiously, confidently, quickly, stiffly 4
hesitantly, nervously, steadily, tentatively, unsteadily 3

awkwardly, casually, determinedly, excitedly, gently, hurriedly, sadly, 2

uncertainly, warily
apprehensively, calmly, despondently, dramatically, dreamily, drunk- 1
enly, furtively, gingerly, gleefully, happily, languidly, limply, noncha-
lantly, pensively, proudly, quietly, resolutely, shyly, smartly, softly,
solemnly, suspiciously, swiftly, thoughtfully, unhurriedly, wearily
As noted in section 3.1, troponyms may be used as a compact way to con-

vey action and thought simultaneously. For “looks” we counted the fol-
lowing: “stares” (f=659), “gazes” (f=180), “peers” (f=178), “glares” (f=66),
“studies” (f=49), “considers” (f=27), “peeps” (f=15), “squints” (f=15), “gapes”
(f=14), “regards” (f=13), “gawps” (f=12), “peeks” (f=11), “admires” (f=7), “leers”
(f=4), “eyeballs” (f=3). Troponyms for “walks” included the following: “steps”
(f=711), “strides” (f=162), “hurries” (f=137), “rushes” (f=122), “wanders” (f=90),
“marches” (f=84), “strolls” (f=61), “creeps” (f=54), “paces” (f=52), “staggers”
(f=48), “glides” (f=45), “stumbles” (f=42), “clambers” (f=35). These trop-
onyms, from “steps” to “admires,” fall across the thought–action continuum.
We have seen that “looks” can also be used in a different sense to
describe how a character appears to be in a particular mental state, often
using an adjective ending in -ed, another easily identifiable form. Table 3
shows a variety of words that are used in this way. Of course, these words
can be used in other contexts and when we count them in the whole cor-
pus we find the most frequent to be “shocked” (f=53), “confused” (f=41),
“puzzled” (f=40), “worried” (f=34), “stunned” (f=34), “terrified” (f=33),
“concerned” (f=32), “troubled” (f=29), “surprised” (f=25). Looking at an
automatically generated list of other words (not ending in -ed) that occur
to the right of “looks” gives the following with their frequencies counted
in the whole corpus: “blank” (f=39), “nervous” (f=30), “awkward” (f=24),
“uneasy” (f=24), “thoughtful” (f=24), and many less frequent words.
The word “smiles” was noted as one of the common actions modified
with -ly words. Consulting WordNet gave a list of other facial expressions; a
similar list could have perhaps been derived from looking at the left context
of -ly words. These were counted in the forms -Ø, -s, and -ing (“smile,” “smiles,”
“smiling”) giving counts of “smile” (f =1,209), “frown” (f =199), “grin” (f =189),
“scowl” (f =29), “smirk” (f =26), “squint” (f =25), “snarl” (f =21), “wince” (f =21),
“grimace” (f =19), “glower” (f =16), “sneer” (f =12), “pout” (f =2), “simper” (f =1).

144 Style
Table 3: Frequencies of word forms ending -ed that follow the word form “looks.”
puzzled 17
confused 15
troubled 11
shocked 10
concerned 7
embarrassed, surprised 6
amazed, disappointed, worried 5
bemused, pleased, stunned 4
annoyed, dazed, perplexed, terrified, unconvinced 3
agitated, alarmed, exasperated, horrified, interested, relieved, unim- 2
pressed, unnerved
aggrieved, amused, ashamed, astonished, baffled, bewildered, bored, 1
captivated, chuffed, defeated, deflated, delighted, determined, disgrun-
tled, disheartened, dismayed, dispirited, distracted, disturbed, drained,
entranced, excited, flattered, frightened, impressed, irritated, moved,
mystified, peeved, perturbed, petrified, saddened, scared, shell-shocked,
startled, strained, stumped, unconcerned
Looking at some of these words in context highlighted phrases such as “smil-

ing with” and “frowning with” are followed by a word referring to a mental
state. Words occurring to the right of “with” include “concern” (f=21), “relief ”
(f =17), “interest” (f =14), “delight” (f =14), “surprise” (f =12), “horror” (f =11),
“fear” (f =10), “shock” (f =10), “alarm” (f =9), “satisfaction” (f=7), “pleasure”
(f =6), “anticipation” (f =5), “worry” (f =5), “emotion” (f =5), “sadness” (f =5).
In turn, inspection of these words in other co-texts pointed to the pattern
“Action in Mental State,” e.g., “stares in amazement.” Words (abstract nouns)
following “in” include “thought” (f=32), “amazement” (f=26), “shock” (f=22),
“disbelief ” (f=20), “horror” (f=20), “alarm” (f=18), “wonder” (f=18), “surprise”
(f=14), “frustration” (f=13), “disgust” (f=10), “dismay” (f=9), “confusion” (f=9),
“fear” (f=6), “awe” (f=6). Some of these words also appear following “look of ”
(f=44). All these words provide valuable information to the listener without
access to the on-screen images about characters’ mental states.

References to body parts can also be used to indicate states of mind. It

was noted in Salway (2007) that words denoting body parts were unusu-
ally frequent in the audio-description corpus. Looking in more detail at
how these words are used, we find the following phrases describing actions
involving eyes.
closes his/her eyes (f =86)

opens his/her eyes (f =40)
eyes widen (f =24)
lowers his/her eyes (f =18)
rolls his/her eyes (f =15)
lifts his/her eyes (f =15)
narrows his/her eyes (f =14)
eyes fill with tears (f =12)
raises his/her eyes (f =8)
eyes glisten (f =8)
eyes flicker (f =8)
eyes meet (f =8)
Less frequent, but perhaps also important in the depiction of characters’

minds, are actions with other body parts, including the following: “bows his/
her head” (f =16) and “hangs his/her head” (f =14), and “puts his/her hand”
(f =39) and “puts his/her arm around” (f =12). Some actions are directly
associated with an emotion: “kiss” (f =235), “crying” and “cries” (f =26),
“weeps” and “weeping” (f =21). More rarely, some audio descriptions refer
to a character’s expression, and describe it as a “pained expression” (f =6),
“serious expression” (f =5), or “puzzled expression” (f =4). There are also a
few instances of “expression” followed by a description of a change: “expres-
sion changes” (f =10), “expression softens” (f =4), “expression becomes” (f =4),
“expression darkens” (f =3).
4 . CLOS IN G REMA RKS
We hope that we have shown how our twin-headed interdisciplinary

approach can contribute in a number of ways to audio description research
and practice, and that we have encouraged readers to explore how they may
use narratology and corpus linguistics for themselves. Here we explained

146 Style
how the thought–action continuum and other narratological concepts can

resolve apparent tensions between only describing what can be seen and
the need for audiences to learn about characters’ mental states, and how
more generally they can help to articulate discussions about what needs to
be described. This points to one potential way in which narratology could
input into the revision of audio description guidelines. We also showed how
corpus linguistics techniques can be used to gain a broad and empirically
grounded overview of audio description practice. Maybe in the future such
techniques could be used to monitor the observance or otherwise of guide-
lines by analyzing samples of audio descriptions as they are produced. In
our research, these techniques allowed us to establish that the provision of
information about characters’ mental states is widespread in the selected
sample of British audio description scripts, which post hoc justifies our
chosen focus.
Of course, our findings do not necessarily apply to audio description in other
countries and/or languages. Analyses of multilingual corpora would help to
determine whether linguistic affordances and cultural preferences affect where
audio descriptions lie on the thought–action continuum (for an example of
multilingual research see Mazur and Chmiel). Further possible studies include
investigating how the positioning of an audio description utterance on the
thought–action continuum is affected by other sources of information about
a character’s mental state that are available to listeners, e.g., from dialogue, and
from cues in the soundtrack. With respect to further synergies between audio-
visual translation, narratology, and corpus linguistics, we note the potential to
work in analogous ways with corpora of subtitles, and ongoing research that is
looking at how patterns induced from a corpus can elucidate some of the narra-
tive functioning of audio description in a data-driven manner (Salway).
NOTES
1. Unless specified otherwise, all frequency counts here relate to specific word forms,
rather than lemmas. We think it is appropriate to use frequency here, rather than relative
frequency, because we are comparing descriptions of the same film.
2. Indeed, the ITC guidelines (ITC 2000) suggest the use of adverbs (point 3.9) and the
use of “specific” verbs (point 3.11).
3. http://www.perl.org.
4. Expressions that can specify word forms and patterns to be searched for in a corpus;
for more examples, see http://www.regular-expressions.info/.

RE F ERE NCES
Anthony, Laurence. AntConc (Version 3.4.2) [Computer Software]. Tokyo, Japan: Waseda
University, 2011. Web. 24 Feb. 2015. http://www.antlab.sci.waseda.ac.jp/.
Biber, Douglas, Conrad, Susan, and Reppen, Randi. Corpus Linguistics: Investigating
Language Structure and Use. Cambridge: Cambridge UP, 1998. Print.
Fellbaum, Christiane, ed. WordNet: An Electronic Lexical Database. Cambridge, MA: MIT
Press, 1998. Print.
Finbow, Steve. “The state of audio description in the UK – from description to narration.”
Perspectives: Studies in Translatology 20.1 (2010): 215–29. Print.
Herman, David, ed. The Cambridge Companion to Narrative. Cambridge: Cambridge UP,
2007. Print.
Ibañez, Anna. “Evaluation criteria and film narrative. A frame to teaching relevance
in audio description.” Perspectives: Studies in Translatology 18.3 (2010): 143–53.
Print.
ITC. “ITC Guidance on Standards for Audio Description.” 2009. Web. 10 Aug. 2013. http://
www.ofcom.org.uk/static/archive/itc/itc_publications/codes_guidance/audio_
description/index.asp.html.
Kruger, Jan-Louis. “Audio narration: re-narrativising film.” Perspectives: Studies in
Translatology, 18(3) (2010): 231–49. Print.
———. “Making meaning in AVT: eye tracking and viewer construction of narrative.”
Kruger, Jan-Louis and Orero, Pilar. “Audio description, audio narration – a new era in
AVT.” Perspectives: Studies in Translatology 18.3 (2010): 141–42. Print.
Matamala, Anna and Orero, Pilar. “Audiovisual translation. When modalities merge.”
Mazur, Iwona and Chmiel, Agnieszka. “Towards common European audio description
guidelines: results of the Pear Tree Project.” Perspectives: Studies in Translatology
20.1 (2012): 5–23. Print.
Palmer, Alan. Fictional Minds. Lincoln and London: U of Nebraska P, 2004. Print.
Remael, Aline and Vercauteren, Gert. “Audio describing the exposition phase of films.
Teaching students what to choose.” Trans. Revista de Traductología 11 (2007): 73–
93. Print.
Ryan, Marie-Laure, ed. Narrative Across Media: The Languages of Storytelling. Lincoln:
U of Nebraska P, 2004. Print.
Salway, Andrew. “A Corpus-based Analysis of Audio Description.” Media for All: Subtitling
for the Deaf, Audio Description and Sign Language. Eds. Jorge Díaz Cintas, Pilar
Orero, and Aline Remael. Amsterdam: Rodopi, 2007. 151–74. Print.
———. “The Computer-based Analysis of Narrative and Multimodality.” New Perspectives
on Narrative and Multimodality. Ed. Ruth Page. London and New York: Routledge,
2010. 50–64. Print.
Snyder, Joel, ed. “Audio Description Guidelines and Best Practices.” Web. 10 Aug. 2013.
http://www.acb.org/adp/ad.html.

148 Style
Vandaele, Jeroen. “What meets the eye. Cognitive narratology for audio description.”
Perspectives: Studies in Translatology 20.1 2012: 87–102. Print.
Vercauteren, Gert. “Towards a European guideline for audio description.” Media for
All: Subtitling for the Deaf, Audio Description and Sign Language. Eds. Jorge Díaz
Cintas, Pilar Orero, and Aline Remael. Amsterdam: Rodopi, 2007. 139–50.
Print.

Palmer - Salway - Audio Description On The Thought-Action Continuum

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Palmer - Salway - Audio Description On The Thought-Action Continuum

Uploaded by

Copyright:

Available Formats

Audio Description on the Thought-Action Continuum

Author(s): Alan Palmer and Andrew Salway

The term audio description refers to the spoken commentary that is

Style 49.2_02_Palmer_Salway.indd 126 22/05/15 12:41 AM

While striking an exceptionally unequivocal tone, these guidelines reflect a

Style 49.2_02_Palmer_Salway.indd 127 22/05/15 12:41 AM

Style 49.2_02_Palmer_Salway.indd 128 22/05/15 12:41 AM

Style 49.2_02_Palmer_Salway.indd 129 22/05/15 12:41 AM

In section 2.1, we introduce some fundamental concepts from the field of

2.1 Why Should Audiovisual Translation Be Interested in Narratology?

Narratology (or narrative theory) is the branch of literary study that is

Style 49.2_02_Palmer_Salway.indd 130 22/05/15 12:41 AM

of the characters in that novel? Readers enter the storyworlds created by

Style 49.2_02_Palmer_Salway.indd 131 22/05/15 12:41 AM

2.2 Analyzing Three Scenes from The English Patient

The broad framework outlined above can now be applied to a comparison

Style 49.2_02_Palmer_Salway.indd 132 22/05/15 12:41 AM

“Katherine catches Almasy’s eye. He is transfixed. Clifton’s smile fades. Katherine

B is more informative than A, even though it is only four words longer,

Style 49.2_02_Palmer_Salway.indd 133 22/05/15 12:41 AM

In this scene, B is noticeably more involved in Katherine’s emotions

Style 49.2_02_Palmer_Salway.indd 134 22/05/15 12:41 AM

In this section, we seek to demonstrate the utility of corpus analysis

Style 49.2_02_Palmer_Salway.indd 135 22/05/15 12:41 AM

audio description provides information about characters’ mental states,

3.1 Further Comparison of Descriptions A and B

This is, at best, a preliminary attempt at a categorization. It does not

Style 49.2_02_Palmer_Salway.indd 136 22/05/15 12:41 AM

­ ata-driven approach may need to be complemented by an approach that

She smiles benignly.

It is unsurprising that the writers of audio descriptions rely on stock or

Style 49.2_02_Palmer_Salway.indd 137 22/05/15 12:41 AM

mental states. These frequently recurring terms may be thought of as “pegs”

Almasy looks thoughtful.

Style 49.2_02_Palmer_Salway.indd 138 22/05/15 12:41 AM

He looks thoughtfully from the hand to the surrounding rocks.

Style 49.2_02_Palmer_Salway.indd 139 22/05/15 12:41 AM

and “glares.” We sourced a list of troponyms of “looks” from the online

Table 1: Frequencies of troponyms of “looks” in descriptions A and B.

Style 49.2_02_Palmer_Salway.indd 140 22/05/15 12:41 AM

be impractical to replicate parts of this analysis on a much larger sample

3.2 Information about Mental States in a Corpus of Audio

It could be argued that the prevalence of information about characters’

Style 49.2_02_Palmer_Salway.indd 141 22/05/15 12:41 AM

“sadly” (f =50), “awkwardly” (f =46), “frantically” (f =42), “cautiously” (f =40),

Style 49.2_02_Palmer_Salway.indd 142 22/05/15 12:41 AM

awkwardly, casually, determinedly, excitedly, gently, hurriedly, sadly, 2

As noted in section 3.1, troponyms may be used as a compact way to con-

Style 49.2_02_Palmer_Salway.indd 143 22/05/15 12:41 AM

Looking at some of these words in context highlighted phrases such as “smil-

Style 49.2_02_Palmer_Salway.indd 144 22/05/15 12:41 AM

References to body parts can also be used to indicate states of mind. It

closes his/her eyes (f =86)

Less frequent, but perhaps also important in the depiction of characters’

4 . CLOS IN G REMA RKS

We hope that we have shown how our twin-headed interdisciplinary

Style 49.2_02_Palmer_Salway.indd 145 22/05/15 12:41 AM

how the thought–action continuum and other narratological concepts can

Style 49.2_02_Palmer_Salway.indd 146 22/05/15 12:41 AM

Style 49.2_02_Palmer_Salway.indd 147 22/05/15 12:41 AM

Style 49.2_02_Palmer_Salway.indd 148 22/05/15 12:41 AM

You might also like

ata-driven approach may need to be complemented by an approach that

3.2 Information about Mental States in a Corpus of Audio