SOUND DESIGN & THE

Ben Denny
1510153@chester.ac.uk

SIGNIFICANCE OF AUDIO
IN COMPUTER GAMES
A case study of Mario Kart 8 and
This essay is going to examine many facets of audio and its uses in games including
its methods and theories surround production, how those methods are successfully
used within the context of a product utilising case study, The significance of digital
audio within the computer games industry as well as various issues including both
legal and ethical, more specifically copyright when relating to publishing digital
audio. This essay will also provide audiovisual examples in the references below.
The first subject this essay is going to examine is the core methods, technologies
and theories surrounding audio in the media, and how the human mind reacts to
sound and music. Many experts such as Carlile believe That audio and music are
the most important game elements in evoking emotional responses from users
there are also many studies that are trying to show that music and audio can be
used to regulate a response to a user. And as game developers, we can utilise this
to generate to put our user in the desired state such as in horror games such as
Resident Evil 7 and many others they utilise music, sound effects, and silence in
order to create a sense of tension for the user which then creates an air of fear
whilst making the user feel fully immersed in the world which will be examined in
further depth later.
(Capcom, 2017)

One of the biggest technological advances that had a significant effect on computer
games audio is the advent of digital audio, digital audio has many advantages
which make it less difficult to create the audio that you specifically would want to.
Some of these advantages are the ability to sequence/multitrack which allows you
to edit compositions on more minute levels with individual instruments in order to
make more specific music and sound effects in order to make variations of audio to
use in methods such as adaptive audio. During the advancement of technology,
there was A constant battle (collins, 2008) between the quality of audio that they
were able to produce and the technology that was available at the time. However,
these constraints had some positive effects of computer games music early on in
the NES era. One of the major ones being Because of their constraints, that older
games could only work with a single chord at a time composers would have to focus
on a strong melody line, which in turn meant that their songs were inherently more
memorable due to its simplicity. (Portnoy, 2012) this demonstrates why some of
the most prominent and memorable computer game songs are from the classic era
of games such as Super Mario Bros and The Legend of Zelda.
Next, this essay is going to examine a method/use of audio which is unique to
interactive mediums such as video games which are adaptive audio and to do this
we will use Nintendo EADs Mario Kart 8 as a case study which contains numerous
excellent examples of the ways in which adaptive audio can be used to convey
different moods and gameplay mechanics to a player both increasing immersion
and excitement but also helping the player succeed in the game

(NintendoEAD, 2014)
A great example of the game is the way that it manipulates the music based on the
current state the player is in according to a wide variety of different factors and
situations the player can be in such as on the track Big Blue, as you reach certain
points on the track more instruments begin to enter the composition making the
song thicker and faster paced as you are getting closer to your goal. Also on all the
tracks when you are in first place, the song increases in speed intending to make
the player more tense as any other racer could pull them out of first with a single
item. Which further increases in speed for all players as they enter the final lap. This
is all to keep this sense of fast paced and tense competitive energy that this game
thrives on. However, this is not the only way the game is enhanced through audio,
at certain invisible points on certain tracks the developers have set it to crossfade
to a specific part of the soundtrack so that it will perfectly match the action
occurring on screen an example being the final lap from Big Blue as presented in
the clip.
A third way in which Mario Kart 8 demonstrates why audio is significant to the
player is its use to help convey mechanics to a player, there are many different
items which have sound effects and other audio techniques applied to them such as
the alarm like noise which plays warning a player that a red shell is heading towards
you. Which gives players time to react to the danger and possibly avoid it if they
have to tools and skill level.
Another feeling a games developer can create immersion which is indispensable for
developers who would like to get players invested in a narrative focused game,
immersion is the sense that players are actually existing within the games universe
or the world. There are many methods that the developer can use in order to
produce this effect in their players, in terms of audio one of the most prominent
methods in games is the use of acousmatic sound which is when the user can see
hear a sound of any type but cannot see the point, object or creature from which it
originated. This can increase immersion in variety of ways such as in a horror game
such as Resident Evil 7, sound of which you are unsure of the origin can create a
sense of the being unsure therefore creating tension within the player which can
they be used to make an effectively terrifying segment an example from resident
evil 7 being hearing the game's antagonists the baker family spouting insane
dialogue from the next room over unsure of where they currently are located. It can
also be used to fake out the player when the Bakers feign discovering the player
putting them further on edge. The developer can also use acousmatic sound in a
more mechanically helpful way in order to guide the player through an environment
such as to find other NPCS or as with the previous example from Resident Evil 7
assisting the players in hiding from the antagonists.
Another great example of the use of acousmatic sound in games whilst being one of
the few examples of diegetic soundtrack in games which is when the soundtrack
you are hearing is coming from within the world of the game and could be heard by
the characters is level Prosper bluff from Supergiants game Bastion in which youre
trying to find an NPC to rescue them and the way you are able to locate them is she
is singing a song which becomes more clear as you are progressing towards them.

(Supergiant, 2012)
As seen in the previous section there are many ways in which immersion can be
given to the player, however, audio can be used more specifically as describing
Ermi and Mayras SCI-Model (2004, p. 7-8) which describes 3 different ways in which
audio can help immerse a user, this is further explored by Huiberts who describes
audiovisually impressive, three-dimensional and stereophonic worlds that surround
their players in a comprehensive manner Construct the sensory experience of the
players which is the first part of the SCI model. This can be seen in many games
such as Battlefield ONE in which the audio fully immersed the user in the chaos of a
world war 1 battle.
The second part of SCI-Model is challenged based immersion which Ermi and Mayra
describe as being fully immersed in the overall suspense of playing this occurs
when players experience a balanced level of challenge and skills when they are
succeeding and advancing in the game and typically involves the use of a player's
motor skills and rapidly responding to stimuli but can also be mental challenges
such as puzzles as well. An example of a product which successfully and
consistently has the player in a state of challenge based immersion is the souls
series in which the balance is so well maintained that a player has to reach a level
of high concentration to play successfully to succeed in these notoriously difficult
games.

(Fromsoft, 2011)
The Third and final part of the SCI-Model is imaginative based immersion which
Huiberts describes as the engagement with the imaginary world and fantasy, game
characters, worlds, and storylines. And can be achieved excellently with audio in
numerous ways such as in the Witcher series by CD Projeckt Red which is based on
the novel series by Andrzej Sapkowski which is in turn based on polish folklore, uses
classical polish folk artists mainly the band Percival as the accompaniments to many
of the songs in Marcin Przybyowicz and Mikolai Stroinskis score giving a sense of
place and realism and therefore immersion to an extremely fantastical world.
Finally, This essay is going to explore some of the problems which a developer could
have when developing sound effects, music and other audio assets for their
product. One of the most major problems being that of Copyright which is a creators
exclusive rights to make copies, license, and otherwise exploit a literary, musical or
artistic work whether printed, audio, video etc. (Dictionary.com, 2007) This means
that if you are not the copyright holder for a sound effect or another audio asset you
would be using in your project you would be unable to monetise the project due to
this being a case for copyright infringement. There are ways to prevent a product
having issues with Copyright the easiest being is creating all of the assets yourself
so that the creator is the owner of all the assets. However if a team is unable to
produce their own assets of which music is mostly like for this to occur with, is you
can license the asset from the owner for a fee which gives you the right to use the
asset within your product, that happens most in games such as Grand Theft Auto
where popular songs are played on the radio within the game and all of it must be
licensed. Sometimes license contracts also request royalties from your products
which mean for each use or per volume sold, or some time period of use or sales
the team will have to continue making payments. However, there are some artists
who offer royalty free assets which means that the creators pay a single fee and
have complete rights to use the asset as much as they would like without having to
continually pay royalties.
 Reference List

Clips
Ryerson University. StudioFeed.(2012, March 16th). Music and the Brain: The Smart
Lab
[Video file]. Retrieved from https://vimeo.com/38640869
Portnow, J. et al. Extra Credits.(2012, February 5th). Extra Credits: Video Game
Music
[Video file]. Retrieved from https://www.youtube.com/watch?v=CKgHrz_Wv6o
Brown, M. Mark Brown.(2014, November 6th). Adaptive Soundtracks| Game Makers'
Toolkit
[Video file]. Retrieved from https://www.youtube.com/watch?v=b0gvM4q2hdI
Guides. GameXplain.(2015, April 15th). Mario Kart 8 DLC: Big Blue Gameplay (60fps
- Full Race)
[Video file]. Retrieved from https://www.youtube.com/watch?v=S02cjEh6RZA
Walkthrough. eXnGamingNetwork.(2011, September 5th). Meeting Zia in prosper
bluff. [Video file]. Retrieved from https://www.youtube.com/watch?v=a3Mrqt0f0VA

Documents, Journals, and Definitions

Huiberts, S. (2010). Captivating Sound: The Role of Audio for Immersion In
Computer Games (Ph.D. thesis). Retrieved from
http://download.captivatingsound.com/Sander_Huiberts_CaptivatingSound.pdf
Ermi. L. Mayra. F (2004). Fundamental Components of the Gameplay Experience:
Analysing Information. Retrieved from
http://citeseerx.ist.psu.edu/viewdoc/download?
doi=10.1.1.103.6702&rep=rep1&type=pdf
Collins, K., (2008). Game Sound: An Introduction to the History, Theory, and Practice
of Video Game Music and Sound Design. Cambridge, Mass.: MIT Press
Mestre, R, M., (2005). Immersion and Presence. CNRS University of the
Mediterranean
Young, M, D. (2012). Adaptive Game Music: The evolution and future of the dynamic
music systems in video games (BA Thesis). Retrieved from https://etd.ohiolink.edu/!
etd.send_file?accession=ouhonors1340112710&disposition=inline
Dictionary.com (2017). Copyright Retrieved from
http://www.dictionary.com/browse/copyright
Kock, C .(2015). Interview: The Witcher 3 Composer Mikolai Stroinski Talks About
How Music Helps Bring The Game To Life Retrieved From
http://www.techtimes.com/articles/53814/20150518/the-witcher-3-wild-hunt-
composer-mikolai-stroinski.htm

Images & Audio

Wikipedia.com (2011) Bastion (Video Game) [Image] Retrieved From
https://en.wikipedia.org/wiki/Bastion_(video_game)
Herrera, K. (2016). Why Resident Evil 7 Will Be Entirely In First-Person Perspective,
According to the Producer. [Image] Retrieved From
http://www.cinemablend.com/games/1525299/why-resident-evil-7-will-be-entirely-in-
first-person-perspective-according-to-the-producer
Amazon. (2011). Dark Souls PlayStation 3 [Image] Retrieved from
https://www.amazon.com/Dark-Souls-Playstation-3/dp/B004NRN5EO
Przybylowicz, M. Et al. (2015). The Witcher 3: Wild Hunt Soundtrack (Full) [Audio]
Retrieved From
https://www.amazon.com/Dark-Souls-Playstation-3/dp/B004NRN5EO