Professional Documents
Culture Documents
xiii
xiv ◾ Introduction
areas of the face at the same time. Consideration should also be given to
the onset and offset timings of expressions that may relate to the inten-
sity of the emotion experienced, thus demonstrating the complexities of
the emotion process. Designers may get away with (albeit benefit from) an
oversimplification of facial expression in anthropomorphic-t ype charac-
ters with a cartoonish, stylized appearance, yet this approach could clearly
not be taken with characters of a realistic, human-like appearance.
Much of the advice that I gave to students was to improve how one
facial expression may develop and then transform to the next expression,
a process that I referred to as expression choreography. However, how does
one define this concept? Furthermore, what would be the implications
of a character’s speech on their facial expression and how that expres-
sion shifted to the next? This difficulty not only in creating but also in
explaining how to represent emotional expression in human-like virtual
characters suggested to me that our current understanding of how to
define and represent facial expression of emotion needed to be improved.
At first, I looked to examples of near-realistic human-like characters fea-
tured in contemporary games and animation to see how this dilemma may
be resolved. Yet it appeared that even professionals working in the anima-
tion and games industry may not have found the answer to this plight.
Advances in technology have enabled animators and video game
designers to pursue a high degree of simulated realism and include real-
istic, human-like characters in animation and games (e.g., in genres such
as action-adventure games and role-playing games). It was intended that
this increased realism would allow the viewer to appreciate the emotional
state of the characters, thus leading to a heightened state of engagement
(and enjoyment) in the game or animation (Doerr, 2007; Hoggins, 2010;
Ravaja et al., 2008). However, rather than accepting these new realistic,
human-like characters, the audience was critical of them (Crigger, 2010;
Doerr, 2007; Hoggins, 2010). Characters that fell below viewer expecta-
tions in terms of their appearance and behavior were associated with the
Uncanny Valley (Mori, 1970/2012). The Uncanny Valley, which proposes
that viewers are generally less accepting of a character as the human-
likeness for that character increases, now exists as a hypothetical bench-
mark for measuring whether a character is perceived as believably realistic
and authentically human-like. The words realism and realistic are used,
for the purposes of this study, to indicate that a designer has intended
that a character will be perceived as authentically and believably human-
like (Sommerseth, 2007). The perceived realism and human-likeness of
Introduction ◾ xv
2007; Gouskos, 2006). This female protagonist was intended to evoke pity
from the audience for her misfortune, yet viewers failed to empathize with
her and were put off by her strange and bizarre behavior. The audience
was also aware of a distinct asynchrony of speech with her lip movement,
which further exaggerated the uncanny for Mary Smith. Her performance
was thus regarded as more comical than seriously thought-provoking.
Quantic Dream returned to the drawing board, and on later release of
Heavy Rain in 2010, despite claims that characters would portray the
smallest details in facial expression, viewers were still left confused as to
the affective state of (the intended to be) empathetic characters featured in
the game (Crigger, 2010; Hoggins, 2010). Even though the graphical fidel-
ity of characters such as the main protagonist, Ethan Mars, had improved,
the audience could still detect imperfections in this character’s facial
expression and speech. The game designers meant for Ethan and other
protagonist and antagonist characters to evoke heightened emotion in
the player so that they may participate fully with the story (Doerr, 2007).
However, awkward and strange facial expressions and a distinct lack of
synchrony of lip movement with speech left some players uninspired and
disengaged with the game (Hoggins, 2010).
Such feedback about characters featured in Heavy Rain provided some
support that the Uncanny Valley phenomenon existed in realistic, human-
like characters, yet this concept still required experimental investigation
(Brenton et al., 2005; Pollick, 2010). Despite various attempts to simulate
human-like facial expression and speech in animation and games (both in
prerecorded and real-time footage), there is general agreement that largely
such performances can fail to be perceived as real and suspend disbelief
for the viewer. As such, the Uncanny Valley is now popular lexicon in the
discourse of animation and games, and the phenomenon now occurs as
a recognized design issue when creating realistic, human-like characters
(see, e.g., Brenton et al., 2005; Doerr, 2007; Geller, 2008; Green et al., 2008;
Ho and MacDorman, 2010; Hoggins, 2010; Plantec, 2007; Pollick, 2010;
Van Someren Brand, 2011; Walker, 2009). Given the increasing impor-
tance of the effective communication of a character’s emotional state to
the viewer so that the viewer may engage with and understand the inten-
tions of that character during the game (or watching a film), I chose to
focus my research on how speech and facial expression of emotion may
influence viewer perception of the uncanny (i.e., trigger experience of the
Uncanny Valley effect) in virtual characters. Specifically, I recognized
that, while facial expression and speech were reported as factors that may
Introduction ◾ xvii
REFERENCES
Azar, B. (2005) “How mimicry begat culture,” Monitor on Psychology, vol. 36, no. 9,
p. 54.
Blakeslee, S. (2006) “Cells that read minds,” New York Times. Retrieved October 31,
2013, from http://www.nytimes.com/2006/01/10/science/10mirr.html.
Brenton, H., Gillies, M., Ballin, D. and Chatting, D. (2005) “The Uncanny Valley:
Does it exist?” Paper presented at the HCI Group Annual Conference:
Animated Characters Interaction Workshop, Napier University, Edinburgh,
September 5–9.
Busso, C. and Narayanan, S. S. (2006) “Interplay between linguistic and affective
goals in facial expression during emotional utterances,” in Proceedings of the
7th International Seminar on Speech Production, Brazil, pp. 549–556.
Crigger, L. (2010) “Bad romance: Love in the time of videogames,” Gamespy.
Retrieved October 31, 2013, from http://www.gamespy.com/articles/108/
1087383p2.html.
Darwin, C. (1872, reprinted 1965) The Expression of the Emotions in Man and
Animals, Chicago: University of Chicago Press.
Doerr, N. (2007) “Heavy Rain devs have ‘conquered’ the Uncanny Valley,” Joystiq.
Retrieved October 31, 2013, from http://www.joystiq.com/2007/12/18/
heavy-rain-devs-have-conquered-the-uncanny-valley/.
Ekman, P. (1979) “About brows: Emotional and conversational signals,” in
Von Cranach, M., Foppa, K., Lepenies, W. and Ploog, D. (eds.), Human
Ethology: Claims and Limits of a New Discipline, New York: Cambridge
University Press, pp. 169–202.
Ekman, P. (1992a) “An argument for basic emotions,” Cognition and Emotion,
vol. 6, pp. 169–200.
Ekman, P. (1992b) “Are there basic emotions?” Psychological Review, vol. 99, no. 3,
pp. 550–553.
Introduction ◾ xxiii
Pollick, F. E. (2010) “In search of the uncanny valley,” Lecture Notes of the
Institute for Computer Sciences. Social Informatics and Telecommunications
Engineering, vol. 40, no. 4, pp. 69–78.
Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S. and Keltikangas-Järvinen, L.
(2008) “The psychophysiology of James Bond: Phasic emotional responses
to violent video game events,” Emotion, vol. 8, no. 1, pp. 114–120.
Sommerseth, H. (2007) “Gamic realism: Player perception, and action in video
game play,” in Proceedings of Situated Play, DiGRA 2007 Conference, Tokyo,
Japan, pp. 765–768.
Spielberg, S. (producer/director) (2011) The Adventures of Tintin: The Secret of the
Unicorn [Motion picture]. Los Angeles, CA: Paramount Pictures.
The Last of Us (2013) [Computer game], Naughty Dog (Developer), Japan: Sony
Computer Entertainment.
Van Someren Brand, N. (2011) “Walking through the valley of life: Motion scanning
as a bridge for crossing the Uncanny Valley,” MA thesis, Utrecht University.
Walker, S. J. (2009) “A quick walk through uncanny valley,” in Oddey, A. and White,
C. A. (eds.), Modes of Spectating, Bristol, UK: Intellect Books, pp. 29–40.
Winerman, L. (2005) “The mind’s mirror,” Monitor on Psychology, vol. 36, no. 9,
pp. 49–50.
Zemeckis, R. (producer/director) (2007) Beowulf [Motion picture]. Los Angeles,
CA: ImageMovers.