Who am I

that acts?
The Use of Voice in
Virtual Reality
Interactive Narratives

Jonathan Barbara1,2 Mads Haahr1

1TrinityCollege Dublin
2Saint Martin’s Institute of Higher Education
Overview Who
speaks?
• Voice in Narratives
• “Who speaks” in Virtual Reality
• Case Studies Who am I
• “Who am I that acts?” - Self-identification that speaks?

Who am I
that acts?
Voice in Narratives

Who speaks?

VOICE

Narrator Narratees
Genettian Narration
Diegetic first person Non-diegetic third person
Homodiegetic Narration Heterodiegetic Narration
• Autodiegetic narrator • Heterodiegetic narrator
• Protagonist • Non-diegetic character
• Homodiegetic narrator
• Non-protagonist
Ambiguity in Narrator’s identification
 Voice and Perspective

Perspective
• Who sees?

Voice
• Who speaks?
Non-focalisation Genettian Perspective

Who sees?
Internal focalisation

External focalisation

Protagonist 3rd parties

Shift of focalisation and narration along time

Internal focalisation External

Autodiegetic narration Homodiegetic narration

Voice, Mimesis, and Gameplay
• ‘Generative narrators’
• Non-diegetic narration serves to direct the mimetic gameplay

Tutorial: non-diegetic HUD Gameplay: non-diegetic HUD

Reference to non-diegetic controls Red Error Messages accompanied by
Red Error Messages Android voice and profile picture
Voice, Mimesis, and Gameplay
• Diegetic voice needs to be loaded with emotional semantics
• in order to help deliver the narrative and cohere it with the gameplay.

Tutorial and Gameplay: minimal HUD

Voice refers to diegetic actions
Text refers to non-diegetic controls
“Who speaks” in Virtual Reality
• Seeking out a diegetic owner of a voice that informs
• Perspective
• Focalisation

• Challenges to immersion
• Non-focalised owner (knows too much!)
• Non-diegetic owner (reminds me I’m not really here!)

• Heightened immersion
• Diegetic voice with external focalisation
Self-Identification
Who speaks?
Who address me?

Who am I that speaks?

Who am I that acts?

Second Person Voice in VR
• The second person ‘YOU’
• Acknowledged presence
• Who am I?
• Through whose aural and visual senses do I perceive the virtual world?
• What is my relationship to the virtual enviroment?
• What is my relationship to the virtual characters?
DelConte’s Narratee identities
• Narratee as The Narrator
• = protagonist
• ≠ protagonist
• Narratee as the Protagonist of the Story
• With a third party narrator
• Narratee as an Onlooker in a protagonist’s story
• Narrated by protagonist
• Narrated by third party
DelConte’s Narratee identities
• Narratee as The Narrator Narrator’s voice mismatch with player’s
• = protagonist breaks immersion
• ≠ protagonist
Using the player’s voice as input
• Narratee as the Protagonist of the Story Does not break immersion
• With a third party narrator
• Narratee as an Onlooker in a protagonist’s story player’s voice with
Augmenting
• Narrated by protagonist real-time spatialisation effects
• Narrated by third party increases telepresence
 DelConte’s Narratee identities
Intradiegetic Narration
• Narratee as The Narrator
• = protagonist
• ≠ protagonist
• Narratee as the Protagonist of the Story
• With a third party narrator
• Narratee as an Onlooker in a protagonist’s story
• Narrated by protagonist
• Narrated by third party
Extradiegetic Narration

Excerpts from CYOA book titled Journey Under The Sea by R.A. Montgomery (1977)
CYOA books evolved...
DelConte’s Narratee identities
• Narratee as The Narrator VR player’s interaction embedded
• = protagonist into diegetic agency
• ≠ protagonist
• Narratee as the Protagonist of the Story
LudoNarrative Dissonance between
• With a third party narrator player agency and author’s story
• Narratee as an Onlooker in a protagonist’s story
• Narrated by protagonist
• Narrated by third party VR players acting as sidekicks (Larsen)
Completing side quests
Without modifying central narrative
Case Studies

The viewer as
a non-protagonist
non-narrating
Character

Different
self-identification
through
different
use of 2nd-person
voice
The Last Goodbye (2017)
• the player accompanies survivor
on a visit to the Majdanek
concentration camp through a
drone-mounted 360 camera
• On-screen and off-screen
persona narrating his memories
The Last Goodbye (2017)
• Never addresses the player with
an absolute ‘you’
• Breaks fourth wall by looking at
camera
• No interactions with the objects
• Swayze-effect
• Camera drone’s shadow
The final scene...

“not in my lifetime, but maybe in yours…”

The Book of Distance (2020)
• Narrator (Randall) takes player
on an imagined journey of
grandpa Yonezo
• Pre-war Hiroshima to Canada
• Narrator often moves off into
the shadows
• Shifting attention to story
VR player as sidekick to Yonezo
• Packing his luggage
• Waving to relatives
• Having passport stamped
• Taking photos
• Building the house
• Sowing strawberries
• Serving strawberries at table

• ... Until WWII happens

VR player as sidekick to Canadian Government
• Lands Yonezo into an internment
camp
VR player as sidekick to Randall, the narrator
• Player as an observer
• Looking at photos
• Reminiscent of Randall’s
unfamiliarity with what
happened after the war
Analysis

The Last Goodbye

• A focus on the player’s
experience rather than the
narrating survivor’s.
• Linear narrative progression
Analysis

The Book of Distance

• Use of second-person ‘You’
• Direct address by the narrator
• Empowered Player’s agency
• Followed by reduction
Suffering Self-Identification
• Who am I that acts?
• Sidekick to Yonezo the grandpa?
• Sidekick to Randall the narrator?
• Sidekick to the Canadian government?
• Which side is the sidekick on?

With no clear intradiegetic narrative role,

Only the extradiegetic role remains
Conclusions
• Self-identification as an
important factor of immersion
• ‘Who am I that acts’ is informed
by Genettian focalisation
• Narrator
• Protagonist
• narratee
• VR player as a virtual
sidekick
• Case studies underserve
self-identification
• Future work can explore any
causal relationship between
• Second-person voice address
of player
• Interactivity offered