CHI 2007

2006 • ACM Student Research Competition AprilApril

22-27,28-May
2006 •3,Montreal,
2007 • San
Quebec,
Jose, CA,
Canada
USA

Speed Sonic Across the Span:

Building a Platform Audio Game
Michael A. Oren
Iowa State University ACM Classification Keywords
1620 Howe Hall H.5.5. Sound and Music Computing.
Ames, IA 50011 USA
moren@iastate.edu Research Problem and Motivation
Boarding schools exist throughout the United States to
provide education for the visually impaired. Teachers
Abstract
and administrators of these schools have bemoaned the
fact that visually impaired students have few options to
We describe the design process and initial user study of
entertain themselves compared to fully sighted stu-
an audio game created for the visually impaired. Until
dents. We initially discussed this issue with students
the advent of 3D graphics games, platform games
and staff at the Indiana School for the Blind and after a
where the player jumps from platform to platform, such
couple of focus group meetings with students; we de-
as Mario and Sonic, [1] were wildly popular. Although
cided to make an audio game in the platform genre.
many audio games have been developed over the past
decade, the platform genre has been all but ignored. To
We chose to create a platform audio game because
fill this gap and to add to the limited choices visually
several of the visually impaired users we spoke with
impaired gamers have, we developed a platform game
had fond memories of playing Super Mario Bros video
that can be also be played via audio-only interface and
game [2]. We also found that very few platform audio
compared it to a traditional audio-visual version.
games exist, despite their abundance in video games
for sighted players.
Keywords
Accessibility, Audio Games, Sonification, User Study,
Conceptually, platform games are very simple—a player
Platform Games
hops on or over objects such as, obstacles and plat-
forms, to reach a goal [1]. However, due to the 2D
spatial nature of platform games, they do not translate
Copyright is held by the author/owner(s).
easily to good audio cues that are vital for an audio
CHI 2007, April 28 – May 3, 2007, San Jose, California, USA
game. We had to create audio cues to represent the
ACM 978-1-59593-642-4/07/0004.
game’s objects, but we also had to find methods to
indicate the spatial relationships (the height, distance,

2231
CHI 2007
2006 • ACM Student Research Competition
and length) of objects as these relationships are critical
to the gameplay.
Our goal was to create a platform game for visually
impaired students to enhance their entertainment op-
tions. At the same time, we saw a chance to investigate
the difficult problem of representing a 2D spatial map
via audio cues alone. Results from this investigation
could be applied to non-game applications, such as en-
coding spatial information (distance) into audio as part
of a multi-modal information system or for use in vir-
tual environments.
Background and Related Work
Audio games have been around for over a decade.
Many “classic” games have been converted to audio
games, such as Space Invaders and Doom [3,4]. In
addition, guidelines have been written regarding the
design of simple audio games [5].
The key issue for sound-based game design is to effec-
tively represent game objects via audio cues. For ex-
ample, objects can be represented by the kind of sound
they would make if they were real, rather than an ab-
stract sound (such as a simple piano chord) [5,6]. Us-
ing this approach as opposed to more abstract auditory
mappings (such as mapping each object to a different
instrument or a different note) allows users to better
recognize and remember the sounds since the user
does not have to learn a new mapping. However, some
types of game objects are not easily connected with
natural audio cues, such as pits or platforms, and those
objects require more abstract sounds the user must
learn and remember [5].
AprilApril
22-27,28-May
2006 •3,Montreal,
2007 • San
Quebec,
Jose, CA,
Canada
USA
Currently available audio games run the gamut from
simple memory games to first person shooters such as
Shades of Doom [4, 5]. While games such as Shades
of Doom and several others have high levels of com-
plexity and require an understanding of spatial relation-
ships, the majority of audio games are fairly simple
maze or puzzle games. More complex games would
require that a more realistic spatial environment can be
presented via audio; however the concept of distance
(or spatiality) ultimately requires a 2D or 3D rendering
of the sound and bi-aural cues. Without these bi-aural
cues, relationships are difficult to determine [5]. In
addition, due to limitations of sound hardware and
software processing, sounds sent through stereo chan-
nel may blend together making it difficult for users to
separate the various channels and accurately judge the
sound properties [5].
This lack of an easy way to use audio cues to provide
spatial awareness is particularly problematic for plat-
form games where multiple objects and their positions
need to be presented to a user while simultaneously
allowing him or her to react to these objects and for
example, to jump over an obstacle. The only example
of an audio platform game that is currently available is
Super Liam [7]. However, Super Liam differs from tra-
ditional platform games in that the action takes place
on a single platform and jumping over obstacles is op-
tional—the main character has a projectile weapon
used to remove obstacles [7]. This simplifies the game
mechanics and reduces the need for the user to have
an accurate understanding of the spatial relationships
of objects within the game.
2232
CHI 2007
2006 • ACM Student Research Competition AprilApril
22-27,28-May
2006 •3,Montreal,
2007 • San
Quebec,
Jose, CA,
Canada
USA

Approach and Uniqueness

While audio games exist in various genres, very little
attention has been given to platform games. In our
initial focus group sessions with visually impaired stu-
dents in boarding schools, we discovered their interest
in playing games in this genre. Our initial research into
audio games found that this particular genre poses
special challenges in design due to the spatial relation-
ship requirements and that it would require an auditory
interface with multiple stationary and moving objects.

Our audio game still lacks the complexity of its graphi-

cal counterparts. However, we tried to include as many
of the classic platform elements as possible. The game
features platforms, jumping and obstacles, such as
dogs that move horizontally, bees that move vertically
(fly), and bottomless pits. It is played on a low-end PC
Figure 1: A screenshot of the game illustrating most of the
or Mac via a gamepad and headphones [8].
important objects.

The game world is divided into four vertical floors which
define the height of the platforms. The platforms may
be floating, so that the “player” (the user’s representa-
tion on the screen, “You”) may travel beneath them.
Other platforms are solid, so that the player will run
into a wall if s/he encounters it and does not jump. Fig-
ure 1 illustrates the primary game objects.
Objects in the game can “emit” an audio cue that en-
codes information about its type. These sound cues are
then modified to provide information about the spatial
relationships between the player and the object (dis-
tance and location), we use left-right panning, change
of pitch and the changes to the repeating-speed of the
sound cue. The audio interface was deliberately left
fairly sparse to avoid clutter and confusion.

Platforms only emit a sound cue when the player is fac-

ing an edge of a platform and if the player is able to
jump on the platform, this simulates a sort of sonar-
return. The tops of floating platforms can only be
jumped on if the player is not walking directly under
the platform, and if the platform is no more than one
floor above the player’s current position.

2233
CHI 2007
2006 • ACM Student Research Competition
The user determines the location of the platforms from
relative audio cues. The platform’s direction is deter-
mined by the way the platform sound is panned: either
left or right depending on the platform’s position rela-
tive to the location of the player. Information regarding
a height difference comes from the platform sound be-
ing pitch bent up (becoming more shrill) or pitch bent
down (becoming deeper), depending on the platform’s
relative position above or below the player. We chose
to have only those platforms emit sounds that are one
level above or at any position below the player (i.e., on
which the user would actually be able to jump) and
omit all other platform cues to avoid confusion.
If the player approaches the edge of a platform other
than the platform the player is on top of, a “platform-
notification” sound is played. The distance from the
player to the edge of the platform relates to the speed
of repetition of the audio cue - with a faster repetition
representing the player being closer to the platform’s
edge.
When the player nears the end of the platform s/he is
located on, an “end of platform” warning will sound to
inform the user that s/he must either jump to another
platform or fall to the platform below. However, if fal-
ling directly downward from the end of a platform
would result in falling into a bottomless pit, a pit warn-
ing sound is played, indicating the need to jump over
the pit or fall into it and start the level over from the
beginning.
Similar to the audio cues for platforms, we decided to
use audio cues that are relative to the position of the
player for representing the position of the enemies
(dogs and bees). However, their sound cues are emit-
AprilApril
22-27,28-May
2006 •3,Montreal,
2007 • San
Quebec,
Jose, CA,
Canada
USA
ted regardless of whether or not the player is facing
towards them. A dog is able to move horizontally and
will start to track the player - the speed and volume of
the dog’s bark becomes faster and louder as it moves
closer to the player. When the dog is very close to the
player, it will start to growl, indicating that the player
must jump now or be bitten and start over.
Sometimes the dog is not even on the same platform
level as the player, in which case, the vertical position
of the dog is indicated by the bark’s pitch - a lower bark
indicates that the dog is located lower, whereas a
higher pitch indicates that that dog is at a higher posi-
tion.
For bees, which only move vertically, a buzzing sound
is panned left or right depending on the side it is lo-
cated on relative to the player. Information about the
vertical position of the bee is again encoded in the pitch
of the audio cue; the bee’s volume will increase with
proximity to the player. The bee will sting on contact
and send the player back to the start of the level. As
the bee uses a steady (buzzing) sound stream, the user
is able to judge when the bee reaches the highest and
lowest points of its path by simply listening to the pitch.
This allows the player to bypass the bee by jumping
over it at its lowest position or to run underneath it at
its highest position.
Results and Contributions
In order to test our application, we conducted a user
study consisting of 18 undergraduate and graduate
students (8 males and 10 females). We worked with
only sighted subjects but blindfolded half of them. The
subjects were randomly assigned to play either the
2234
CHI 2007
2006 • ACM Student Research Competition
audio-only version (blindfolded) or a traditional video-
game version (audio-visual.
With a researcher present, subjects first read the in-
structions and then played through ten increasingly
complex training levels to familiarize themselves with
the game mechanics and the audio cues. Subjects then
played through the seven levels of the actual game.
The subjects’ game performance was scored based on
the speed of level completion (faster being better) and
the number of times they were sent back to the begin-
ning of the level (more times being worse). After com-
pleting the main game, the subjects filled out an exit
survey in which they rated their ability to recognize the
various objects, spatial relationships, etc.
Statistical analysis of the results showed there was no
mean difference between the audio-only and the audio-
visual subjects in terms of relevant background experi-
ence. Nor was there a mean difference between the
performance of men and women on any of the variables
collected. On average, the subjects played computer
(video) games on occasion. The subjects use computers
very frequently, while they infrequently use programs
that represent visual objects with auditory signals. On
average, the subjects had some familiarity with plat-
form games.
We asked the subjects in what way they typically use
audio while playing computer games: 5% of subjects
simply ignore the sounds; 61% of subjects report that
the sound enhances the realism of the game, but they
do not actually use it in a functional (guiding) way;
28% use sound to warn them of critical game events;
such as an incoming attack; and 39% percent of sub-
jects use sound to obtain information about enemies
AprilApril
22-27,28-May
2006 •3,Montreal,
2007 • San
Quebec,
Jose, CA,
Canada
USA
and objects behind them. Only 17% of subjects had
any previous experience with audio interfaces (any
software system where graphical objects are repre-
sented by audio cues, such as a screen reader) and
these subjects preferred to obtain information from
audio by a combination of sound and speech signals.
Subjects found the dogs, the bees and the pit very easy
to identify based on their sound, while platforms were
more difficult to recognize. This may have to do with
the more abstract sounds given to platforms.
The majority of subjects in the audio-only group indi-
cated that they were able to determine both the verti-
cal and horizontal location of objects via the sound.
Some subjects indicated problems with recognizing the
vertical axis (pitch). Conversely, those in the audio-
visual group did, on average, find the sound beneficial
in locating objects along the horizontal plane but did
not find audio cues beneficial in locating objects along
the vertical axis. This difference may indicate that the
sound cues did not provide any added benefit to what
the players could already see on the screen.
The average audio-only subjects, and some audio-
visual subjects, also used audio cues in determining the
distance, direction, height, and type of objects. How-
ever, some subjects indicated that audio cues did not
help determine the speed of objects nor their length,
which is to be expected given that we did not provide
any dedicated cues for speed or length.
Although the number of restarts did not vary signifi-
cantly between groups, it took the audio-only group, on
average, nearly twice as long to complete levels as the
audio-visual group. Whether this is due to their inexpe-
2235
CHI 2007
2006 • ACM Student Research Competition
rience with audio interfaces or, perhaps, problems with
correctly using the auditory cues provided is unclear at
this point.
This project allowed us to take what, graphically, is a
very simple game and translate it into a game accessi-
ble to the visually impaired and blind. Despite the
slightly modified rules and, for blindfolded gamers, an
unusual approach, the majority of participants found
the game enjoyable. Several participants, who re-
quested to try out the alternative version of the game
after completing the exit survey, from each of the
Acknowledgements
Thanks and appreciation goes to my research advisors,
Chris Harding, Iowa State University, for his support
and recommendations for improvements to this project
and Terri Bonebright, DePauw University, for her
knowledge and recommendations regarding the usabil-
ity study. We thank the students and staff of the Indi-
ana School for the Blind, particularly Al Lovati who
served as our direct contact. We would also like to
thank Carol Farrenkopf, for her advice and guidance in
working with the visually impaired and blind. Finally,
we would like to thank Kyle Shipley who worked on the
first iteration of this project with us.
Part of the research has been funded by NSF grant IIS-
534681.
Citations
[1] Platform Games, Wikipedia.
http://en.wikipedia.org/wiki/Platform_game.
AprilApril
22-27,28-May
2006 •3,Montreal,
2007 • San
Quebec,
Jose, CA,
Canada
USA
groups even commented that they found the audio-only
version more challenging and more enjoyable.
This not only showed that an audio game requiring
complex 2D spatial relationships can be built and
played with a relatively good success rate, but it also
shows that audio games can also be enjoyed by sighted
gamers. The audio techniques we used to create this
2D game may also be useful as a teaching tool for help-
ing visually impaired students understand 2D maps and
floor plans. It may be possible to apply some of our
techniques to existing projects, such as BATS to im-
prove current audio maps [9].
[2] Super Mario Bros., Wikipedia.
http://en.wikipedia.org/wiki/Super_Mario_Bros.
[3] McCrindle, R.J., Symons, D. Audio Space Invaders.
In Proc. Intl Conf. Disability, Virtual Reality, & Assoc.
Technology, University of Reading, UK (2000), 59-65.
[4] Shades of Doom.
http://www.gmagames.com/sod.html.
[5] Gärdenfors, Dan. Designing Sound-Based Com-
puter Games. In Cybersonica Symposium (2002), 1-7.
[6] Gaver, William W., Norman, Donald A. Everyday
listening and auditory icons, Ph.D. Dissertation, Univer-
sity of California, San Diego, 1988
[7] Super Liam, AudioGames.net.
http://www.audiogames.net/db.php?action=view&id=s
uperliam.
[8] GamePad, Wikipedia.
http://en.wikipedia.org/wiki/Gamepad
[9] BATS, University North Carolina.
http://www.cs.unc.edu/Research/assist/bats/features.s
html
2236