• Embed Doc
  • Copy Link
  • Readcast
  • Collections
  • CommentGo Back
Download
 
 HEAT SEEKER
– AN INTERACTIVE AUDIO-VISUALPROJECT FOR PERFORMANCE, VIDEO AND WEB
Nuno N. Correia
 Aalto University, School of Art and Design, Media Lab Hämeentie 135 c, FI-00560 Helsinki, Finland 
ABSTRACT
 Heat Seeker
is an audio-visual project, which has been released in different formats: performance, video, and website. Inthis paper, it is contextualized with similar projects that combine music and visuals. The motivation and aims behind
 Heat Seeker
are then presented. The main objective is to combine visuals with sound in an electronic music performance,creating an engaging hypermediated experience for the audience. A description of the project and its developmentfollows, including project extensions to different platforms, such as the Web. Conclusions are reached regarding theaccomplishment of the initial aims, which are only partially achieved, particularly in the areas of flexibility of the projectand coherence of content. Paths for future developments are then outlined, in terms of additional project extensions andadditional projects.
KEYWORDS
Music visualization, audiovisual tool, audiovisual performance, new media art, net art, interaction design
1.
 
INTRODUCTION
 Heat Seeker
is an audio-visual project by Video Jack (a collective composed of the author and AndréCarrilho), developed between 2003 and 2009. It was showcased as audio-visual performance between 2004and 2008 – initially in Portugal, and later in festivals in Poland, Germany and Russia. A CD/DVD based onthe project was released in 2006. In the following two years, the
 Heat Seeker
DVD was screened in festivalsin the UK, Brazil, China and France. A web version was released in 2009, allowing users to interact with thevisuals, creating their own “music videos” to the tracks from
 Heat Seeker
.Next,
 Heat Seeker
will be contextualized with similar works, followed by a description of the project andits development.
2.
 
CONTEXTUALIZATION
The
 Heat Seeker
project is related to similar projects combining sound and visuals. This audio-visual juxtaposition has a long tradition in history.The relation between music and image has been studied throughout the centuries. Already in ancientGreece, philosophers considered that there was a correlation between the musical scale and the rainbowspectrum of hues (Moritz 1997). In the 17th century, Isaac Newton tried to connect sound oscillations to theircorrespondent light waves. Several artists have tried to create a “total art work” fusing different art formstogether, such as Richard Wagner with his concept of “gesamtkunstwerk” (Wagner 1849). Explorations inthe first half of the 20th century, notably within creative movements such as Bauhaus and Futurism, took theconcept of “synthesis of the arts” further. In the 1920s, Oskar Fischinger and Walther Ruttman created“visual music” films in Germany – a combination of tinted animation with live music (Moritz 1997).Fischinger was an inspiration to a younger generation of visual music artists that emerged in the mid 20
th
 century, such as brothers John and James Whitney (Moritz 1995).The development of electronic technologies inspired other authors to explore new ways of synthesis of the arts. Ascott named this new convergence “gesamtdatenwerk”, updating Wagner’s concept (Ascott 1990,
IADIS International Conferences Computer Graphics, Visualization, Computer Vision and Image Processing 2010Visual Communication 2010: Creative Industries, Photography and Culture 2010and Web Virtual Reality and Three-Dimensional Worlds 2010243
 
p. 307). Many of these projects explore an integrated audio and visual expression. These projects often “standin the tradition of kinetic light performance or the
visual music
of the German abstractor and painter OskarFischinger” (Paul 2003, p. 134). Artists such as Golan Levin (often in collaboration with Zachary Lieberman)have explored interconnected audio-visual creative expression, in works such as
Audiovisual Environment Suite
(1998-2000). Some of these artists create audio-visual instruments and “sound games”. In 1999, JohnKlima created
Glasbead 
, an “online art work that enables up to 20 simultaneous participants to make musiccollaboratively via a colorful three-dimensional interface” (Tribe and Jana 2007, p. 54). In 2005, ToshioIwai’s
 Electroplankton
was released for Nintendo DS, a group of ten different interactive audio-visual gamesthemed around cartoon plankton, building upon past artistic work by Iwai. Also in 2005, Sergi Jordà and histeam at Universitat Pompeu Fabra created
 Reactable
, a multi-user electro-acoustic music instrument with atabletop tangible user interface.
 Reactable
has dynamic visual-feedback capabilities: “a projector (...) drawsdynamic animations on its surface, providing a visual feedback of the state, the activity and the maincharacteristics of the sounds produced by the audio synthesizer” (Kaltenbrunner et al 2006, p. 1).
 Heat Seeker
has similarities with pre-digital works such as Oskar Fischinger’s visual music works, andalso with recent digital playful audio-visual projects such as
 Electroplankton
.
3.
 
METHODOLOGY, MOTIVATION AND AIMS
 Heat Seeker
came about from discussions about electronic music, VJing and audio-visual performancebetween the author (a musician and programmer) and André Carrilho (an animator and illustrator). Theauthor was dissatisfied with the visual element of his electronic music performances, and was interested inusing projected motion graphics as a performance complement. André Carrilho was also interested in VJingand in combining his animation work with music. Both shared a mutual interest in electronic music,illustration and clubbing culture. They decided to develop work together under the name Video Jack.The research presented in this paper was conducted by the author, as developer and user of 
 Heat Seeker
,by means of a practice-based methodology. It is therefore part of an “investigation undertaken in order togain new knowledge partly by means of practice and the outcomes of that practice” (Candy 2006, p. 1).
3.1 Combining of Music and Visuals
The main question
 Heat Seeker
addresses is: how to combine visuals with sound in an electronic musicperformance (restoring a visual element that is lacking in laptop-based music performances), creating anengaging hypermediated experience for the audience? Video Jack also aim to explore other channels topresent the project, beyond performances.Both images and sound embody movement, as Nicholas Cook states. Consequentially any alignment of music and moving image that “reaches a threshold of similarity between the two” can create a “transferenceof kinesthetic qualities” between one and the other. Cook calls this a “kind of ventriloquism” (1998, p. 78).The combination of image and sound, asserts Cook, “contextualizes, clarifies, and in a sense analyses themusic”, activating “a new, or at any rate a deepened, experience of the music” (1998, p. 74).
 Heat Seeker
 aimed to reach this “new and deepened” experience.According to Nicholas Cook, media such as “music, texts, and moving pictures do not just communicatemeaning, but participate actively in its construction” (1998, p. 261). Music has not only meaning but alsopotential for meaning, which is fulfilled in relation with the context in which it is received. Meaning resides“not in musical sound, then, nor in the media with which it is aligned, but in the encounter between them”(Cook 1998, p. 270). Having created the visuals and music of 
 Heat Seeker
in articulation with each other,Video Jack hoped to provide the elements for the construction of a coherent audio-visual meaning.
3.2 Achieving Flexibility and Expression
Another objective of the project is to create a tool for manipulating visuals that has a similar flexibility as amusic instrument, and that could allow for the same kind of improvisation and expression. Automated audioreactivity was not a priority in the development of 
 Heat Seeker
– Video Jack were interested in exploring theuncertainty of human reactions. This would ideally lead to a greater variety of results.
ISBN: 978-972-8939-22-9 © 2010 IADIS244
 
Sergi Jordà, as constructor of musical instruments, aims for “instruments that are enjoyable to play andthat mutually enhance the experience when playing with other musicians”, and “that will surprise (…) asmuch as possible, that will keep revealing little hidden secrets at every new listening” (Jordà 2005, p. 4).With
 Heat Seeker
, Video Jacked aimed for the same degree of flexibility, surprise and variation of result inits visual side, enhancing the musical side, with which it establishes a dialogue.
3.3 Displaying the Act of Content Manipulation
An additional aim of 
 Heat Seeker
was to make the act of manipulating the visuals apparent to the audience,similarly to how the audience views a musical instrument being played live in a performance (an additionalparallel to musical instruments). The combination of the visual content with the visualization of contentmanipulation, in articulation with the music, should ideally result in an engaging experience for the audience.According to Austerlitz, the enjoyment of music has always been linked with the experience of “watchinga performer physically produce musical sound”. The performer’s body language has been a fundamentalaspect of the music experience. The rise of radio and mechanical reproduction of media in the 20
th
centurychanged this scenario. Music became a “commodity”, possible to be “disembodied” from the performers(Austerlitz 2007, p. 11). Parallel with these technological advancements, efforts were made to “reunite theseparated segments of the musical experience”, merging sound and image, and creating a new art form, torealize “Wagner’s dream of 
gesamtkunstwerk 
” (Austerlitz 2007, pp. 11-12).This “reunification of segments of the musical experience” was thus one of the objectives of 
 Heat Seeker
.In laptop-based electronic music performances, the visual impact of physical musical manipulation is usuallylimited. The performer typically employs a limited range of subtle gestures, using a mouse (or track pad) andkeyboard, occasionally complemented with small hardware controllers. The impact of these gestures isdifficult to discern by the audience.
 Heat Seeker
aimed to reunite sound with the visual element of performance, by displaying to the audience the construction and manipulation of visual content, in reaction tothe music. In other words,
 Heat Seeker
aimed to match the transparency of live music performance with non-electronic instruments.Therefore,
 Heat Seeker
shares resemblance with works such as Emergency Broadcast Network’s
Telecommunications Breakdown
, as described by Bolter and Grusin: “the Emergency Broadcast Network’sCD-ROM conveys the feeling that we are witnessing, and in a way participating in, the process of its ownconstruction (…) by emphasizing process” (Bolter and Grusin 2000, p. 54).
3.4 Exploring Additional Project Spin-Offs
Video Jack aimed to explore additional project extensions, additionally to performances. Other projectextensions explored were a video “spin-off” (“music videos” of each track distributed by DVD, online, and infestival screenings) and an interactive online version. This way,
 Heat Seeker
could obtain a larger exposure,reach different audiences, and also satisfy different preferences of their audiences in their roles regarding theproject. These roles could be varied: witnesses to real-time interaction, in the case of performance; viewers of a linear “music video” version; or users themselves, by accessing the interactive web version.
4.
 
DEVELOPMENT AND DESCRIPTION
The development of 
 Heat Seeker
involved two different stages: software development and contentdevelopment (music and animations). These two stages will be presented next, together with a description of the software. The animated content of 
 Heat Seeker
is also contextualized with other works.
4.1 Software Development
Due to the author’s background in programming (particularly Adobe Flash and ActionScript), developing acustom tool for audio-visual performance had the potential to allow for a more personal and flexibleapproach. Discussions in 2003 led to the development of an application for controlling digital animation to
IADIS International Conferences Computer Graphics, Visualization, Computer Vision and Image Processing 2010Visual Communication 2010: Creative Industries, Photography and Culture 2010and Web Virtual Reality and Three-Dimensional Worlds 2010245
of 00

Leave a Comment

You must be to leave a comment.
Submit
Characters: ...
You must be to leave a comment.
Submit
Characters: ...