Visual Communication

http://vcj.sagepub.com

Designing for the unexpected: the role of creative group work for emerging interaction design
paradigms
Antti Pirhonen and Emma Murphy
Visual Communication 2008; 7; 331
DOI: 10.1177/1470357208092323

The online version of this article can be found at:

http://vcj.sagepub.com/cgi/content/abstract/7/3/331

Published by:

http://www.sagepublications.com

Additional services and information for Visual Communication can be found at:

Email Alerts: http://vcj.sagepub.com/cgi/alerts

Subscriptions: http://vcj.sagepub.com/subscriptions

Reprints: http://www.sagepub.com/journalsReprints.nav

Permissions: http://www.sagepub.co.uk/journalsPermissions.nav

Citations http://vcj.sagepub.com/cgi/content/refs/7/3/331

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 visual communication

ARTICLE

Designing for the unexpected: the role of

creative group work for emerging
interaction design paradigms

ANTTI PIRHONEN
University of Jyäskylä, Jyväskylä, Finland
EMMA MURPHY
McGill University, Montreal, Canada

ABSTRACT
Interaction design for new technological environments relies on the
tradition of human–computer interaction (HCI). With roots in the 1980s, HCI
design paradigms often reflect the setting in which the user is an office
worker in front of a desktop computer. As computational power can now be
embedded in almost any type of product, the desktop setting has lost
much of its relevance as a starting point for interface design. In particular,
interfaces for wearable computing challenge designers to look for com-
pletely new approaches to interaction design. In this article, we propose a
method in which the ideas for new creative forms of interaction design are
triggered through panel work. This method draws on an underpinning
theoretical framework from structural semiotics that emphasizes the
holistic nature of design.

KEY WORDS
design • interface • multimodal • narrative • participant • user-centred

INTRODUCTION
Wearable computing or wearable technology is an example of a technology
that is integrated as part of our everyday life, as opposed to a desktop
workstation in an office or home environment. ‘Wearable’ as a concept is a
more concrete idea when compared to, for example, pervasive computing,
which can relate to technologies broadly assigned ubiquitous qualities (see
e.g. http://www.pervasive2008.org/). However, an exploration of literature
related to wearable computing shows that it can, in fact, be conceptualized in
an amazingly large number of ways. It covers the range from highly visible

Copyright © 2008 SAGE Publications (Los Angeles, London, New Delhi and Singapore:
http://vcj.sagepub.com) /10.1177/1470357208092323
Vol 7(3): 331–344 [1470-3572(200808)7:3; 331–344]

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 technological prosthetic devices, such as head-mounted Virtual Reality
applications, to the invisible suture of nanotechnologies into bandages and
clothing.
In this article, an attempt is made to define a creative design method
for wearable computing, as an area that may involve new, possibly unfore-
seen, ways of utilizing personal technology. The aim of this study is to
establish a methodology for the design of products and user interfaces in
terms of human needs and available technology. The building blocks for this
approach are user-centred design, creative group work and a semiotic
perspective to user interface design – all relatively new methods in science
and engineering design practice. We describe this methodology by reference
to a design case study, which concerns the design of a multimodal web
browser for visually impaired users.

CONCEPTUALIZATION AND DESIGN

Previous empirical evidence illustrates that usage cultures, by which we mean
uses of technology and of applications by users that were not foreseen or
predicted by designers, emerge despite technological designs. For example,
SMS (Short Message Service) technology was intended to be a minor feature
in mobile phones, but now text messaging is for many users the primary way
of communicating with a phone. The success of this very simple technology
surprised the developers. The opportunity to send short messages with a
small, portable device created user communities for whom text messaging
was the main means of communication (Sillence and Baber, 2004). The idea
of regularly typing messages with an extremely unusable keypad, originally
intended for dialling telephone numbers was not envisaged by designers.
This example illustrates that the phone designers had not understood an
essential aspect of the user and clearly indicates that there is a gap between
the ideas of the designer and the user. This gap has been identified and
discussed with particular reference to wearable computing as a new
technology. Thompson (2007) highlights the fact that designers need to
develop a cognitive vocabulary in order to understand ‘the relationships
between people and the idea of technology – as an idea’ (p. 42).
Understanding the user is a key issue in design. In user-centred design
(UCD), numerous methods have been developed to aid this difficult process.
However, all empirical methods, whether qualitative (i.e. methods related to
ethnography) or quantitative (i.e. controlled laboratory tests), presuppose
design ideas, which are then exposed to empirical research. The crucial
questions for interface design are: Where should the initial ideas come from?
Can the user be involved in the initial design and development of the
technology and its applications – rather than simply test those technologies
and applications after they have already been designed?
In the context of wearable technology, there is no specific set of
expertise necessary to formulate initial design ideas. Wearable technology

332 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

encompasses such a wide area that many consumers have some experience of
the design issues involved. Therefore, we propose the use of panels consisting
of non-expert participants to brainstorm design ideas. This strategy stresses
the idea that creativity is not a privileged quality but can be seen as an
essential quality of human nature (Leonard and Swap, 1999). Using ordinary
people as a source of creative ideas is the cornerstone of the design method
proposed in this article.
In this design methodology, ‘the user’ and ‘the context of use’ are the
core design elements rather than conventions and predetermined technical
constraints. Further, both ‘user’ and ‘context’ are understood as semiotic, or
meaning-making, rather than realist concepts.

RICH USE SCENARIO

Semiotic analysis has proven appropriate when applied to a range of
symbolic systems, including film, photography, theatre and music. Recently,
it has also been used for the design and analysis of User-Interfaces (UIs).
When applying semiotic analysis to UIs, crucial elements include both
individual or discrete design elements and the overall or holistic design – as
well as the relationship between the two. In the case of desktop applications,
the whole can be argued to be the layout of the Graphic User Interface
(GUI). Each detail of the GUI can be considered or analysed in the context of
the whole. For example, when adding a push-button to a form field, a web
designer considers the size, location and visual appearance of the push-
button in terms of the overall layout of the current window.
So we might ask how we analyse a smart device that does not have a
large display but is a tiny gadget in the user’s pocket. Do we analyse only the
device itself or do we include the context in which the device is used – and if
so, how much of that context: the device and its possible containers (e.g.
pocket, purse) and/or conditions of use (from home to noisy streets and
public transport to work environments where sound may be an issue). This is
a crucial question when discussing the use of mobile applications, such as
wearable computing. It is possible to argue that the appropriate unit for
analysis in wearable computing is the context of use. However, if the
application can be used anywhere, then the notion of ‘context of use’ is still
not sufficiently specific to serve as the basis for analysis or design. For
variable contexts of use, a more tangible concept is required.
We propose the idea of a rich use scenario as a framework for design.
Use scenarios – descriptions of use – have been widely used in the design of
complex systems. Traditionally, use scenarios are prepared in order to
illustrate typical use cases. Usually, several scenarios are needed to attain the
required generic coverage. The related character, the user in the use scenario,
is often described at a level that covers a large proportion of likely user
categories. The background of the character may be described at a general
level, without providing details. For example, age, education, socio-

Pirhonen and Murphy: Designing for the unexpected 333

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 economical background, cultural background, may be formulated using
broad terms, like age group or level of education.
The rich use scenario, as proposed here, is very different from
traditional use scenarios. The rationale behind traditional use scenarios is to
make the implicit explicit, i.e. to describe what exactly happens when an
application is in use. Such scenarios are intended to reveal features which
otherwise would remain hidden. The underlying motivation is to design
products which would be usable for as many user groups and in as many
contexts as possible. However, the purpose of a rich use scenario is not to
cover all kinds of users and situations. A rich use scenario is a vivid story
depicting a credible person in a certain context. A rich use scenario should
help listeners (in the case of narration) identify themselves with the character
in the story. The story should also make it possible to become immersed in
the life of the character and so to predict uses and associated contexts of use
that would otherwise not be predicted by designers.
Previous practical approaches to the design of user interface elements
have illustrated that stories are useful for triggering and focusing design
ideas. Barrass (1996) has previously investigated the use of stories to inspire
design ideas for auditory displays. Bødker (1999) has investigated the use of
extreme characters and even caricatures in her use scenarios to evoke creative
ideas for groups of users and designers. More recently, Franinovic et al.
(2007) have explored participatory approaches to the interaction and design
of everyday sounding objects for the purpose of sound design.

Qualitative vs quantitative methodology

In the proposed design method, the purpose of the use scenario is not to
cover all possible usages of an application or technology. The purpose of the
scenario is to trigger creative design ideas while maintaining the discussion
on a certain context and user character. The story, or rich use scenario,
describes a person in a unique situation, although the use of an application is
obviously in the focus of the story. Gaps occur at appropriate points in the
story, replacing user interface elements such as audio or haptic effects. This is
where user feedback is required.
It might be argued that using one single story and concentrating on
the life of one person (or possibly a couple of people if more than one main
role is appropriate to the story) is a limitation on the validity and usefulness
of the outcome – that is, of the application designed by the methodology. Is it
possible to assume that the specific design that works in the unique setting
described in the story, would work with different characters and contexts?
The answer is that it is not possible to make such an assumption. However, it
is also true that if we try to design for everyone, we finally do not design for
anyone.

334 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

THE USER PANEL SESSIONS
The central idea of the user panel session is that the rich use scenario – not
the application itself – is the basis for design. For the purposes of the design
task described earlier, the rich use scenario was introduced to the design
panel as a radio play. The task of the panel was to design sound effects for
this story. The following steps outline the radio-play design method
(Pirhonen et al., 2006; Murphy et al., 2006).

Stage 1 Prepare a task description about the functions of the application.

Stage 2 Prepare a user description, based on a vision of a plausible user. The
fictitious character should not be designed to have generic qualities,
as the aim is not to cover as many users as possible but to create a
lively, inspiring character.
Stage 3 On the basis of stages 1 and 2, write a short story in which the
interaction between the character and the application plays an
important role. The perspective taken is that of the character – one
should remember that there are many things other than the
application in his or her life and mind. In the story, leave blanks or
pauses for user feedback (the interface elements to be designed).
Stage 4 Organize a design panel session with four to five panellists. Start the
session by reading the use scenario, with a brief pause in the place
of each element to be designed. Having read the story, discuss it at a
general level. Then return to the story, and read again the sentence
that includes a blank space for a user interface element. Ask the
panellists to try and describe what kind of feedback would be
appropriate. Proceed in a similar way with each element to be
designed. Record the session.
Stage 5 Implement the panellists’ ideas of the appropriate feedback.
Stage 6 Organize a second session with different people. This time, use the
draft interface elements (implemented in stage 5) when reading the
story. In other respects, follow the stages of the first session.
Stage 7 Analyse the reactions and new ideas of the second session. Modify
original feedback and create new designs as suggested by the second
panel.
Stage 8 Organize a third session with a different set of participants. Using
the feedback from stage 7, present the use scenario to the third
panel and ask participants to choose interface elements at relevant
points in the story.

This process is represented in Figure 1.

Pirhonen and Murphy: Designing for the unexpected 335

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 Figure 1 Radio-play method.

C A S E S T U D Y: D E S I G N I N G S O U N D S F O R A
M U LT I M O D A L W E B B R O W S E R
In order to evaluate the design theory and method described earlier, a
practical evaluation of the proposed method was carried out. Three design
panel sessions were arranged, with different participants each time, and the
nature of the session varied according to the phase of design. Each session
involved four or five panellists and the duration of each session was
approximately one hour.
The initial aim of the sequence of sessions was to create an iterative
design pattern.

336 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

● The first panel was intended as a brainstorming session – no sounds were
implemented and panellists were encouraged to describe effective
sounds for the use scenario.
● In the second panel, sound samples were presented to participants based
on the ideas presented in the first session. These draft sounds were
designed with the intention that they would be elaborated on during the
second session.
● Finally, these further developed sounds were discussed and evaluated by
a third panel.

Thus, each user panel had a different role in the development of sounds. The
first panel began with event descriptions, giving initial sonic ideas and
principles, while the second session involved working with the first versions
of the sounds. The final session involved making assessments and choices of
modified sounds. (A detailed presentation of the practical study and results
can be found in Pirhonen et al., 2006 and Murphy et al., 2006.) For the
purposes of this discussion, the main features relating to the design method
are now described.

Application to a multimodal web browser

Figure 2 User interacting with a web page

through the multimodal web browser
consisting of audio (speech and non-speech
sound) and haptic feedback (via the
Logitech wingman force feedback mouse).

We begin by outlining the steps involved in the application of the method to

a multimodal web browser (see Figure 2).

Stage 1 Task descriptions and the function of non-speech sounds were

implemented. An auditory interface specifically designed for a
multimodal browser plug-in (Yu et al., 2005) to convey spatial
information on a web page to visually impaired internet users was
used as a case study to test the design method.
Stage 2 A user description, based on a vision of a plausible user was created.
The use scenario for this design study was written from the
perspective of a visually impaired user.
Stage 3 Rich use scenario: the use scenario applied in this study describes a
visually impaired character buying a music file online using the
multimodal plug-in. In the use scenario designed for this case study,

Pirhonen and Murphy: Designing for the unexpected 337

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 the character is introduced as a young visually impaired student
and the scenario describes the character’s mood, the technology he
is using and the sounds in the environment around him.

It was one of those mornings, which Kenny would have preferred to

skip . . . He did not have a screen reader or any other special tools
designed for the blind, but had coped reasonably well to date with the
help of little plug-ins his former girlfriend had installed on his
computer. These plug-ins provided haptic and audio cues to help find
and locate graphical objects in a user-interface . . . as he scrolled
through the playlist to cover the dull sounds of Monday morning:
doors, toilets, showers and all kinds of household appliances
produced in a block of flats create an enormous symphony when
people wake up and leave for to work.

The scenario describes the user’s movements in the process of finding and
buying a music file using the multimodal plug-in. Task descriptions are
punctuated with spaces for possible sounds.

Kenny typed in the address of the site and soon heard the sound
[sound 1] that indicates that the page had successfully opened. At the
start page of the online shop, called Cheaphits, Kenny moved the
mouse across the page, from left to right, then down. He was already
familiar with the tactile and audio feedback and soon got an overall
impression of the page. He was especially happy with the sound that
guided him towards the links [sound 2] – it attracted the mind like a
magnet and made the hand move the mouse towards the link area.
A similar kind of magnetic effect led the mouse towards images
[sound 3] . . .

. . . On this page, all ads appeared to be images, so it was quite easy to

distinguish them from the useful information because of the clear
sound [sound 4] indicating that the mouse was on the image. Another
sound [sound 5] told Kenny that he had reached the link area.

Stage 4 First panel: a design panel session was organized with four panel-
lists. The session began by reading the use scenario, keeping brief
pauses in the place of each sound. Having read the story, the story
was discussed at a general level. Panellists were asked to try to
describe what kind of sound would be appropriate at various points
in the story/use scenario. Participants described their ideas for
sounds by referring to specific examples of distinctive timbres or
sounds from television or software programmes. For example,
participants described the sound of an object such as a glass filling
to depict a web page loading.

338 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

Stage 5 The panellist’s ideas of the appropriate sounds were implemented,
based on the ideas of the first panel.
Stage 6 Second panel: a second session was organized with five different
people. This time, the draft sounds (implemented in stage 5) were
used when reading the story. Panellists commented on the sounds
that were implemented and were encouraged to give suggestions for
new sounds. For example, users commented on the timbre and
sound ideas of implemented sounds.
Stage 7 The reactions and new ideas of the second session were analysed.
The original sounds were modified as suggested by the second panel
and new sound ideas were implemented.
Stage 8 Third panel: a third and final session with a different set of
participants was carried out. Sounds from stage 7 were presented to
the third panel as part of the same use scenario. During this session,
panellists were presented with a choice between sounds imple-
mented based on the previous panel’s suggestions. They reacted to
the sounds that were implemented and suggested modifications for
final sounds.
The final sounds were implemented in the auditory interface and evaluated
separately with visually impaired users. Through the practical imple-
mentation of the design method, a number of issues were highlighted. These
are discussed in the next section in relation to each panel session

METHODOLOGICAL ISSUES

User Panel 1, the brainstorming session: Although, at this stage of the process,
panellists were only required to brainstorm ideas, it would be useful
during this session to include a tool for testing sound ideas to help
users to create specific sound ideas. It can be difficult for panellists
to describe sound ideas in words, especially if they are not familiar
with the relevant vocabulary for describing audio parameters. The
introduction of real sounds could focus the panellists’ ideas and
sound descriptions at this initial brainstorming stage. Even the
provision of simple instruments could allow users to communicate
sounds to each other more effectively. A recent study by Droumeva
et al. (2007) has illustrated how ‘low-tech’ instruments such as
kazoos, claves and shakers can be effective tools for generating
creative sound ideas using basic parameters such as pitch, rhythm
and amplitude.
User Panel 2, draft sounds designed: When draft-quality sounds were
presented to the panel, they were immediately criticized by
participants. In the discussions, it became clear that the main

Pirhonen and Murphy: Designing for the unexpected 339

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 trigger for the negative reaction was not caused by the basic idea of
each sound (formulated in the first panel). The negative reaction
turned out to be an emotional one – the rough-cut sounds were
simply unpleasant to listen to. This negative reaction prevented the
panellists from elaborating on the sounds constructively. Instead,
they presented new, alternative ideas as a basis for subsequent
sound versions, rather than building from those suggested by the
first design panel. Therefore, in order to enable constructive,
iterative development of each sound idea, implementations of first
versions of sounds by the designer should sound polished rather
than draft-like. However, the provision of low-tech sound
generation methods or simple instruments such as those discussed
earlier could also solve this problem. If users could control the
‘draft’ sounds themselves, issues of quality could be overlooked to
focus on the basic ideas and parameters of the sounds.
User Panel 3, implementation: The experience of implementing the third
user panel revealed that it was quite difficult for the researcher to
co-ordinate playing sound choices while simultaneously attempting
to chair the sound design discussion. A solution to this could be to
create a dramatized recording of the use scenario with sounds
included as effects. In this idea, the use scenario could resemble a
radio play. Various versions of the recorded scenario could include
different sound solutions that the panellists could choose from.
Furthermore, by recording the use scenario, a more dramatic story
could be realized to encourage the users to identify with the user
character in the scenario.

The central challenge in the use of the proposed method is to generate a

suitable atmosphere for creative motivation among panellists. Otherwise the
default expectation of the panellists seems to be that they are supposed to
design sounds for an application. However, as stressed earlier, the task of the
panellists in this approach is not to design sounds for an application but
rather to design (or choose in the third session) sound effects for a radio-play
story.
Rich use scenarios may take many different formats. The most
common format is a written story, but it can also be, for example, a video. In
our method, we used a radio-play format. There were a number of reasons
for this choice. First, our task in both cases was to design the audio elements
of a user interface for visually impaired users. This choice enabled us to
engage with sound as a design element. Second, we were concerned with
developing a narrative scenario that could be shared by all participants in the
design process. A radio play enabled us also to utilize a story-telling mode of
audio presentation in our design process. Furthermore, the aim of the design
methodology was to trigger creative ideas, and the use of radio plays is

340 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

considered more effective for this purpose than, for example, an audio–video
presentation such as television. There is a substantial body of research that
illustrates that radio is superior to television for the process of creative
thinking (see e.g. Greenfield and Beagles-Roos, 1988; Greenfield et al., 1986;
Singer and Singer, 1979; Valkenburg and Beentjes, 1997). Although this
empirical investigation mainly concerns school children, creativity is not
necessarily age dependent. Some studies have indicated development of
creativity from childhood to adulthood (Wu et al., 2005).
A written format for presenting the use scenario was also considered
as a method for evoking creative thought. However, for the needs of the
proposed design method, a radio-play format proved more appropriate (see
Pirhonen et al., 2007) for the reasons given earlier. Critically, with a radio
play, the semiotic elements to which listeners are exposed are relatively
constrained/confined, ensuring uniformity of experience.

I M P L I C AT I O N S O F A P P LY I N G T H E R A D I O - P L AY
METHOD TO A WEARABLE DESIGN STUDY
The design case presented in this article describes the design of non-speech
audio signs. However, a modified version of the method has already been
used to design haptic feedback (Kuber et al., 2007). In addition, the approach
is appropriate to the design of large wholes like product concepts, which are
relevant when discussing the design of wearable technology. Sounds as well
as haptics are relevant modalities in the interaction between the user and a
wearable, intelligent device. A rich use scenario in the form of a radio play is
capable of giving creative insights to the role of personal technology in
everyday life.
The radio play has a major role in orientating the attention of the
panellists. It became evident early in the design sessions that if discussion
centres around the technical detail of the application, the panellists start to
design the application rather than the sound effects for the radio play as
intended. The same would certainly apply to the design of wearable
technology. The story should be about a person and his or her life, not about
the application. The challenge of the author of the manuscript or the
designer is to relate the story close enough to the application in order to
obtain appropriate feedback for the actual design. Furthermore, the focus on
the rich use scenario provides a holistic approach to design. This is
particularly relevant for the design of elements, whether through the visual,
auditory or haptic modalities, in order to provide a sense of coherence. In the
design of feedback from wearable interfaces, the fact that the environment
and context of use is changing provides even more need to create a coherent
design solution and a rich use scenario can help to achieve this.
While chairing the panel sessions, it is tempting to systematically go
through details to be designed by asking direct questions to individual
panellists. It is much more difficult to encourage free discussion among

Pirhonen and Murphy: Designing for the unexpected 341

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 panellists. A factor that will affect the fluency of the discussion is the
personality types that make up the panel. It is important to aim at creative
group work, in which all available resources are focused on allowing the
group to brainstorm.
The outcome of the iterative design panels resulted in draft sounds for
the case study presented in this article. These were then implemented and
integrated into the interface. It is proposed that the same could be achieved
for feedback cues for wearable devices. While the role of the panellist is to
create raw interface element ideas and later to react to draft implemen-
tations, the designer is required to analyse the raw transcripts from the panel
and interpret the designs and reactions of the participants to suit the
application design.
Another notable advantage of the radio-play method for designing
elements for wearables is the support it can provide for conceptualizing the
use context. Through the radio play, listeners can become immersed in an
environment, which may be completely new for them. Many of the contexts
of use as well as the usage cultures of wearable computing are unforeseen or
at least unexpected. Immersing users in the world in which the wearable
application will be in active use could provide a fruitful basis for creating
ground-breaking product concepts, user interfaces or single interface
elements.

CONCLUSION
The design methodology presented in this article can provide designers with
the appropriate tools for communicating abstract and intangible design
concepts, including the design of feedback for wearable devices. The
intention of this user panel design method is to trigger creativity within a
design panel so that just one designer does not choose sounds for an
interface based on personal preference or ad hoc choices. Involving a panel of
designers that are removed from the system design process generates a level
of objectivity that is more likely to create effective design solutions. However,
it is the focus on the rich use scenario that marks this approach as different
from other participatory design methods.
The purpose of the rich use scenario is rather to generate creative
input that focuses discussion on the details of the system. The level of detail
included in the description of character and environment in the use scenario
in this method can help to contextualize interface elements as well as
inspiring creative design. The concept of rich use scenario provides a means
for designers to encode their own ideas into the design while still achieving a
level of group confirmation early in the design process. Rather than basing all
design decisions on the preferences of a single designer, it is proposed that
designs should be developed by triggering creativity from within a group to
inspire imaginative design.

342 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

REFERENCES
Barrass, S. (1996) ‘EarBenders: Using Stories about Listening to Design
Auditory Interfaces’, Proceedings of the 1996 Asia-Pacific Conference on
Human–computer Interaction, 25–8 June, Singapore, pp. 525–38.
Beyer, H. and Holtzblatt, K. (1998) Contextual Design. San Francisco: Morgan
Kaufmann.
Bødker, S. (1999) ‘Scenarios in User-Centred Design: Setting the Stage for
Reflection and Action’, in Proceedings of the 32nd Hawaii International
Conference on System Sciences, 5–8 January, Vol. 3, p. 3053.
Canny, J. (2006) ‘The Future of Human–Computer Interaction’, ACM Queue
4(6): 24–32.
Droumeva, M., de Castell, S. and Wakkary, R. (2007) ‘Investigating Sound
Intensity Gradients as Feedback for Embodied Learning’, Proceedings
of the 2007 International Conference for Auditory Display, 26–9 June,
Montreal, Canada, CD-rom format.
Franinovic, K., Visell, Y. and Hug, D. (2007) ‘Sound Embodied: Explorations
of Sonic Interaction Design for Everyday Objects in a Workshop
Setting’, Proceedings of the 2007 International Conference on Auditory
Displays, 26–9 June, Montreal, Canada, CD-rom format.
Greenfield, P. and Beagless-Roos, J. (1988) ‘Radio vs. Television: Their
Cognitive Impact on Children of Different Socioeconomic and Ethnic
Groups’, Journal of Communication 38(2): 71–92.
Greenfield, P., Farrar, D. and Beagles-Roos, J. (1986) ‘Is the Medium the
Message? An Experimental Comparison of the Effects of Radio and
Television on Imagination’, Journal of Applied Developmental
Psychology 7(3): 201–18.
Kuber, R., Yu, W. and McAllister, G. (2007) ‘Towards Developing Assistive
Haptic Feedback for Visually Impaired Internet Users’, Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems, 28
April to 3 May, San Jose, California, pp. 1525–34. New York: ACM Press.
Leonard, D. and Swap, W. (1999) When Sparks Fly. Boston, MA: Harvard
Business School Press.
Lumsden, J. and Brewster, S. (2003) ‘A Paradigm Shift: Alternative Interaction
Techniques for Use with Mobile & Wearable Devices’, Proceedings of
the 2003 Conference of the Centre for Advanced Studies on Collaborative
Research, Toronto, 6–9 October, pp. 197–210. Toronto: IBM Press.
Murphy, E., Pirhonen, A., McAllister, G. and Yu, W. (2006) ‘A Semiotic
Approach to the Design of Non-speech Sounds’, in D. McGookinand S.
Brewster (eds) Proceedings of the 2006 International Workshop Haptic and
Audio Interaction Design, 31 August to 1 September, Glasgow, Vol. 4129 of
Lecture Notes in Computer Science, pp. 121–2. Berlin: Springer Verlag.
Pirhonen, A., Murphy, E., McAllister, G. and Yu, W. (2006) ‘Non-speech
Sounds as Elements of a Use Scenario: A Semiotic Perspective’,
Proceedings of ICAD 2006, June 20–3, London, CD-rom format.
Pirhonen, A., Tuuri, K., Mustonen, M.-S. and Murphy, E. (2007) ‘Beyond
Clicks and Beeps: In Pursuit of an Effective Sound Design

Pirhonen and Murphy: Designing for the unexpected 343

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009

 Methodology’, in I. Oakley and S. Brewster (eds) Haptic and Audio
Interaction Design: Proceedings of Second International Workshop,
HAID 2007, 29–30 November, Seoul, Korea, Lecture Notes in
Computer Science, pp. 133–44. Berlin: Springer Verlag.
Sillence, E. and Baber, C. (2004) ‘Integrated Digital Communities:
Combining Web-Based Interaction with Text Messaging to Develop a
System for Encouraging Group Communication and Competition’,
Interacting with Computers 16(1): 93–113.
Singer, J.L. and Singer, D.G. (1979) ‘Come Back, Mister Rogers, Come Back’,
Psychology Today 12, March: 56, 59, 60.
Thompson, S. (2007) ‘Mind the Gap: Technology as Soma’, AI & Society
22(1): 37–44.
Valkenburg, P.M. and Beentjes, J.W.J. (1997) ‘Children’s Creative Imagination
in Response to Radio and Television Stories’, Journal of
Communication 47(2): 21–38.
Välkkynen, P., Ailisto, H. and Pohjanheimo, L. (2006) ‘Physical Browsing’, in A.
Vasilakos and W. Pedrycz (eds) Ambient Intelligence, Wireless Networking,
and Ubiquitous Computing, pp. 61–81. Boston, MA: Artech House.
Wu, C.H., Cheng, Y., Ip, H.M. and McBride-Chang, C. (2005) ‘Age
Differences in Creativity: Task Structure and Knowledge Base’,
Creativity Research Journal 17(4): 321–6.
Yu, W., Kuber, R., Murphy, E., Strain, P. and McAllister, G. (2005) ‘A Novel
Multimodal Interface for Improving Visually Impaired People’s Web
Accessibility’, Virtual Reality, Special Issue on Haptic Interfaces and
Applications 9(2): 133–48.

BIOGRAPHICAL NOTES
ANTTI PIRHONEN is a senior researcher. His doctorate was in educational
sciences, although currently he is preparing a second thesis in computer
science at the University of Jyväskylä. He has been a scientific leader of
several user–interface sound-related projects since 1999.
Address: Department of Computer Science and Information Systems, PO Box
35, FI–40014, University of Jyväskylä, Finland. [email: pianta@jyu.fi]

EMMA MURPHY is currently a post-doctoral research fellow at the

Multimodal Interaction Lab, McGill University, Montreal. She is an honours
graduate of Trinity College, Dublin, where she studied Music and
Philosophy. Since graduating from TCD in 2000, her career path has followed
from full-time involvement in research and development in industry to
academic study. In 2007, she completed her doctoral thesis at the Sonic Arts
Research Centre in Belfast.
Address: Multimodal Interaction Lab, McGill University, School of
Information Studies, 3459 McTavish, Montreal, QC, Canada H3A 1Y1
[email: emma.murphy@mcgill.ca]

344 Visual Communication 7(3)

Downloaded from http://vcj.sagepub.com by loredana ivan on October 16, 2009