You are on page 1of 4

Trilobyte: a Sensory Sound Design Experiment

Vincent Olislagers
Student, Interaction Design, Malmö University
Student, Communication & Multimedia Design, Media & Design Academy Genk
vincentolislagers@gmail.com

ABSTRACT time. We were interested in gaining more insight into the process
This paper covers the development of a physical computing of sound design. Time itself defines sound, its pitch, and its form.
prototype with as goal sound synthesis through context-aware Sine waves of higher pitched sounds have a shorter wavelength
computing from a critical design perspective. It discusses the than low pithed sounds – this means that there is less time
development and outcomes of “Trilobyte”, an electronic sound between each consecutive phase. For this reason we chose sound
experiment through which the user could generate an abstract in relation to time as our area of interest when starting the
auditory representation of the context wherein the object is used. conception process.
I also have a personal interest in the field of critical design which
Categories and Subject Descriptors I further wanted to look into within the scope of this project. I
H.5.2 [Human Computer Interaction]: User Interfaces – Haptic chose to develop an object that would augment a digitally
I/O, Input devices and strategies, Interaction styles, Prototyping. generated signal with analogue data that represented properties of
the physical world.
General Terms
Measurement, Performance, Design, Experimentation
2.2 Project Scope and Context
Keywords The first week of work for the “Trilobyte” project was spent on
Place-specific computing, Context-aware computing, Tangible conceptualization and acquiring more technical knowledge using
User Interfaces, Sound design, Critical Design, Play Arduino and the Processing programming language [3]. In the
second week we started developing the physical prototype. At this
point we had made the decision of whether or not to make sound
1. INTRODUCTION design one of our focal points.
“Trilobyte” was developed during the physical computing course
of the Interaction Design master program at Malmö Högskola
within a timeframe of two weeks. It is a product of the final 2.3 Ideation and Conceptualization
assignment which was described as “create a game looking into A big part of the conceptualization process consisted of
time, history, and life”. During the development brainstorming brainstorming and rapid prototyping. From this process emerged
and rapid prototyping were used to come to a prototype that the idea of creating an object that would slowly charge itself over
explores the combination of analogue and digital data by time and that would be drained of its energy when interacted with
generating place-and-time dependent sounds. by its user. We also thought about the notion of having time not
only impact the properties of the object but also the interaction
between the object and its user. In a sense, the interaction between
2. THE DESIGN PROCESS instrument and player is also affected by time. For example, in
order to play the trumpet one needs to force breath through the
2.1 Design Goals instrument. This process is finite; the trumpet player will become
When we started the development phase of what later would turn exhausted after extensive play. Then, there is also the notion of
out to become “Trilobyte” we had a certain set of goals in mind. time directly correlating to the mastery of the instrument in that it
Firstly, we wanted to explore a haptic way of interacting with takes years to become a skilled musician with most traditional
technology by applying our new found knowledge of the Arduino instruments. It is no coincidence that the object through which the
prototyping platform [1] in the development of the prototype. artist creates music is called an „instrument‟, as it is a highly
Tangible interaction was one of our objectives in designing an specialized tool designed to help its user perform an action in a
immersive and playful experience. We wanted to break away from skillful manner. [4]
existing interaction models which allow only for a limited
immersion in this type of experience. Ishii and Ullmer [2] argue “Instruments are created out of tools by the skill of the operator”.
that the interactions between people and cyberspace are still too (Beynon, M. Nehaniv, C & Dautenhahn K. 2004, p.2),
much governed by the traditional desktop model, there is a divide In this sense the user and her skill level defines to what extent an
between the physical world we act in and the virtual realm we object can be considered an instrument, be it musical or otherwise.
interact with. We thought this notion was worth exploring as part of our critical
Secondly, on a conceptual level we wanted to focus on developing design approach in developing the concept. With the previously
an object which would somehow be impacted by the passing of stated considerations in mind, we experimented in a series of
design iterations focused on sound synthesis from electronic powered via external battery pack. The Arduino board was
sensory measurements. positioned near the outer edge of the other case so it could be
connected to a computer via an USB cable to enable serial
communication.
3. THE CONCEPT This development process was to a great extent influenced by the
3.1 The Metaphorical Meaning resources which were available to us. Originally we wanted to
The name „Trilobyte‟ is a portmanteau of the words trilobite and develop the outer shell from a single sheet of vacuum formed
byte. A trilobite is an extinct marine creature with a segmented plastic. Unfortunately our university doesn‟t possess the
body. It represents both the physical world as well as equipment needed for this procedure. We were also unable to
ephemerality in the form of mortality. In using the metaphor of have our design produced by a local company in the short
the trilobite we take a look at life from the perspective of time; all timespan that was available to us. Instead the enveloping structure
living things are meant to die eventually. The word trilobite stands surrounding the electronics elements was made out of plastic
for „having three lobes‟; its carapace is divided into three sections. sheets which were salvaged from four A4 sized semi-transparent
The outer shell of the “Trilobyte” prototype follows this three- document binders. The design for this structure was made using
layered segmentation, making it visually similar to its extinct the 3D development software Rhinoceros. [5] It was then printed
counterpart. on paper and traced on the polypropylene surface of the binders.
We then cut the material into the desired shapes and used a glue
A byte is a unit of digital information which is used in the gun to unify the segments into the body of the prototype.
communication between the various hardware components of the
concept. The concept aims to explore the interflow and
convergence of the physical and the metaphysical – the digital 3.4 Interaction with the Prototype
realm by augmenting digitally generated sound with data from the
By bending the prototype one could give input to the system and
physical world.
interact with it. When holding each of the exterior cases of the
outer shell in one hand the user could bend the prototype at the
middle section in order to give input to the system and thus
influence the sound stream. As the object was bent, the flex sensor
in the centrepiece read a higher value. This value was then used in
the sound synthesis process, increasing the amplitude of the
digitally generated sine wave. This procedure was noticeable
through a more „violent‟, fluctuating edge to the sound as if the
object was calling out to its user in pain. Bending the prototype
back to its neutral straight position lowered the value of the flex
Figure 1. The trilobite used as a metaphor for the sensor back towards zero which resulted in the decreasing of the
Trilobyte prototype amplitude of the sine wave and results in a more normalised, calm
sound. Decreasing the amount of light that was registered by the
prototype made the generated sound more vivid and distorted.
We had also chosen to have the form and shape of object more
This was representative of the philosophy that the “Trilobyte”
resemble that of an animal than a tool to support our critical
was a nocturnal creature which would become more active during
design angle. Corpus from nature is used by humans for many
night time.
occasions, including the act of music making. For instance, pan-
flutes are made from cut dried reed and some sea shells are used
as an „ocarina‟ which is basically an exotic type of flute.
3.5 Use Qualities
When defining the use qualities and values of the concept we
envisioned the contexts in which the prototype could be
3.2 Functional Properties applicable. One such scenario focussed on the notion that many
“Trilobyte” read analogue data from the material world and uses it
people like to visit live music performances when they can enjoy
to digitally synthesize audio waves in order to produce place
the artist‟s music from the comfort of their own homes. There is
specific sound. It is an electronic sound experiment through which
something mesmerizing about live music, part of the experience
the user can generate an abstract auditory representation of the
for both the artist and her audience is the sense of „being in the
context wherein the object is used. This is done by generating a
moment‟. This sense of finiteness adds a special feeling to the
stream of unique, constantly changing sound which is dependent
occasion. Benjamin describes such a feeling of finiteness as
on user input and lighting conditions, temperature and magnetic
“aura” within the context of art. He argues that in the age of
field of the space wherein the prototype is operated.
mechanical reproduction that which withers and is fleeting is
authentic and that which can be reproduced loses authenticity [6]
3.3 Material Properties Every instrument has its own „sound‟. No electrical guitar sounds
The prototype of the “Trilobyte” was covered by an outer shell the same. This is a direct result of the components used in the
which consisted of two triple layered polypropylene cases which instrument and the craftsmanship with which it is assembled. In a
were connected. These cases held the electronic components that same fashion each concert sounds different because of various
made up the interactive part of the prototype. One of the cases external agents which influence the performance. Think for
held the Arduino I/O board and a small breadboard with some example about the weather conditions, live sound mixing and
sensory components. The other case held a small vibrating motor mental and physical states of the band members which all directly
impact the sound. Messeter describes how the field of interaction The carrier frequency for the sound was synthesized by an
design with its focus on the user experience is becoming more and algorithm that amalgamated the bending value of the prototype,
more reliant on the notion of place and its social and cultural the environmental temperature and a seeded randomly generated
contexts. [7] number into a signal. The harmonicity of the signal was produced
Our prototype also invoked improvised playful interaction from by a series of calculations using the brightness of the space
its audience. During the development phase and the final wherein the prototype was operated. Lastly the modulation index
presentation several people tried to make different sounds with the was also produced by using the bending value of the flex sensor as
object. Some people turned off the light in the room to see how it input. The „energy‟ level of the prototype would affect the gain
influenced the sound. Others covered the exterior of the object (volume) or the sound which was rendered by this procedure. As
with their hands. One person shone a flashlight on it in an attempt the object was interacted with over time, its energy level would
to unravel the logic behind the generated sound. There were also lower which in turn would slowly fade from sound to silence.
people who tried to warm up the shell with their hands or tried to
heat it up by breathing through the dorsal openings in the shell.
What was curious to see is that the prototype was interacted with 5. CONCLUSION
more like an unfamiliar life form than an inanimate object. In a 5.1 Reflection and outcomes
sense it had a will of its own, changing and halting its „voice‟ Working on the “Trilobyte” prototype resulted in an intense but
when it experienced the interaction as uncomfortable or too informative two week learning and testing trajectory. Over the
extensive. course of this project I gained new insights on the subject of
sound design and developed an array of new technical skills
ranging from rapid prototyping with the Arduino platform to the
4. THE IMPLEMENTATION process of sound synthesis with the visual programming
4.1 Hardware and Software Implementation environment Max/MSP. My personal learning goals had also been
The prototype consisted of one Arduino I/O board which had been met. Time as a variable was both influencing the interaction with-
wired to a heat sensor (NTC), light sensor (LDR) and a flex and the output from the prototype. I had also learned a lot about
sensor. Furthermore it read the fluctuations in the magnetic field the process of digital sound creation through doing various
on one of its own analog input ports in order to generate a stream Max/MSP tutorials and by reading up on audio synthesis theory.
of true randomly generated physically read data. The The sound design experiment itself had yielded mixed results. On
microprocessor on the Arduino module was flashed with a script an interaction level it had a less favorable outcome than expected.
that continuously put these readings into variables and sent these The chosen interaction of bending the object turned out not to be
variables over a serial connection from the Arduino to a laptop the most obvious and ergonomic manner of operation. At first
running the Max/MSP software environment. [8] In Max/MSP the glance it was not clear to the user how to interact with the
sensor readings where used as part of a mathematical algorithm in prototype or even if to interact with it at all. However, after the
the process of frequency modulation synthesis to generate sound. \ bending interaction was explained the form of this interaction was
clearly understood as a metaphor for the sine wave manipulation
that was taking place in the sound synthesis process. The notion of
place-specific sound generation was also appreciated by both our
fellow students and tutor.

5.2 A Critical Design Perspective


It was interesting to see how such objects like the “Trilobyte” can
challenge some preconceptions about sound and space and their
ephemerality. In a way “Trilobyte” gave its own critique on the
process of music making as it reflected its environment in the
sound it produced when it was operated.
Our application of place-specific sound generation served mostly
an artistic use in this instance. One can see the “Trilobyte”
prototype as a critical design artifact in the sense that it demands
experimentation from its user in order to give meaningful output
but that, in time, this interaction also prevents the user from
further interacting with it. The user is allowed to learn how to
operate the instrument but never to master it. In a sense it
challenges some preconceptions about time and space.

Figure 2. Effects of user interaction with the prototype on


the discrepancy of the pitch of the generated audio signal
6. ACKNOWLEDGMENTS
Baris Serim for his unfathomable patience and perseverance and
his indispensable input during the project. Furthermore I would
like to thank Sveta Suvorina for giving us useful comments and
insights on many occasions and Alex Cotoranu for his witty
remarks. I also thank the whole of the IDm 2010 class for the
great study atmosphere and for making my adventure in Sweden
so worthwhile. I thank Mette Agger Eriksen, Amanda Bergknut
and Tony Olsson for their valuable feedback. I also thank Niels
Hendriks, Steven Malliet and Jan Louis de Bruyn from my home
university for enabling me to have this wonderful time in Sweden
during my Erasmus exchange.

[5] The 3D CAD software Rhinoceros. http//www.rhino3d.org


7. REFERENCES (Accessed in Dec. 2010)
[1] The Arduino electronics prototyping platform.
http//www.arduino.cc (Accessed in Dec. 2010) [6] Benjamin W. 1936. The Work of Art in the Age of
Mechanical Reproduction. Published in Zeitschrift für
[2] Ishii, H. Ullmer, B. 1997. Tangible Bits: Towards Seamless Sozialforschung
Interfaces between People, Bits and Atoms. Published in the
Proceedings of CHI '97, March 22-27 [7] Messeter, J. 2009. Place-Specific Computing: A Place-
centric Perspective for Digital Designs. International Journal
[3] The Java based Processing programming environment. of Design, Vol 3.No. 1 www.ijdesign.org School of Arts and
http//www.processing.org (Accessed in Dec. 2010) Communication, Malmö University
[4] Beynon, M. Nehaniv, C, Dautenhahn K. 2001. Cognitive [8] The visual programming environment Max/MSP.
Technology: Instruments of the mind. Proceedings of the 4th http//www.cycling74.com (Accessed in Dec. 2010)
International Conference CT. August 6-9