Introduction

Node based VJ System

ProZeuxis will be a compositing system to generate visual effects for enhancing live
performances of any type. The system will be flexible and easily allow the user to create
expressive visuals that incorporate aspects of the performance that will drive the effects;
including live beat detection, MIDI input, and motion tracking. Using these features, my
concentration will be to facilitate improvisation on the part of the visual performer, or VJ, as well
as the performers that their visuals are supporting. To do this, ProZeuxis will be based on an
intuitive, tangible user interface that provides fluid controls for linking generative visual effects to
the live sources of audio and video data. This interface will be created by mainly using a
ReacTable(reacTable) surface on which a user can create compositions of linked nodes. The
interface will be designed to incorporate interaction through the use of tangible markers with
affixed fiducial symbols as well as multi-touch, finger input.
The ProZeuxis system will seek to improve on the performance aspect of improvisation
when compared to other DIY and commercially available VJ systems that also use multi-touch
and tangible interface components. These other multi-touch systems focus on controlling visuals
much in the same way that existing hardware or software systems have in the past. They create
a virtual counterpart interface to reflect these methods, and extend the functionality onto a multi-
touch screen, and in many cases, couple this with tangible controller knobs and sliders as well.
The layout of these systems, however, still revolve around the existing methodologies of creating
a live visual composition by layering banked video clips, and applying effects through the use of
variously controlled parameters. ProZeuxis, while subscribing to a lot the same ideas as these
systems, will also try to depart from the way in which a composition is created in a live setting.
Generative visuals that make use of ProZeuxis' uniquely data rich environment will replace the
tired visuals of manually layered video clips with predicable effects and mix modes.
Although setting up for a ProZeuxis performance may include the necessary steps of
configuring nodes that analyze audio and video data before hand, using these will provide a
resource that can be propagated to many visual nodes within a visual compositions. This mantra
of computing once and reusing many times over, will be a common theme in the design
consideration that will make ProZeuxis truly able to augment any type of performance visually.
Approach using connectible modular nodes

Input data for Tracking / Audio detection shared

Focus on Improvisation / Expression from VJ and Performer shared

pg. 445 Arnheim - Thus, we define expression as modes of organic or inorganic behavior
displayed in the dynamic appearance of perceptual objects or events. The structural properties of
these modes are not limited to what is grasped by the external senses; they are conspicuously
active in the behavior of the human mid, and they are used metaphorically to characterize an
infinity of non-sensory phenomena: low morale or t he high cost of living, the spiraling of prices,
the lucidity of argument, the compactness of resistance.

pg. 455 Arnheim - If expression is the primary content of vision in daily live, the same should
be all the more true of the way the artist looks at the world. The expressive qualities are his
means of communication. They capture his attention, they enable him to understand and
interpret his experiences, and they determine the from patterns he creates.
pg. 151 Significantly, gestalt therapy contrasted spontaneity both to the automatic and to the
conscious act. Spontaneity was characterized by a flexible awareness turned to the gratification
of organic needs. “Spontaneity is that seizing on, and glowing and growing with, what is
interesting and nourishing in the environment.”
pg. 155 In Gestalt Therapy, Goodman valorized painting and poetry as synecdoches of how a
healthy individual should function in the social-animal-physical field. In writing about the artist's
spontaneous creativity, he described a process specific to the “plastic dialog” that was being
pioneered by his friends and contemporaries...His distinction between spontaneity and
automatism identified a crucial theme of the postwar culture of spontaneity, He insisted:
The important part of the psychology of art is not in the dream or in the critical consciousness, it
is... in the concentrated sensation and in the playful manipulation of the material medium. With
bright sensation and play in the medium as his central acts, the artist then accepts his dream
and uses his critical deliberateness...The artist is quite aware of what he is doing...he is not
unconscious in his working, about neither is he mainly deliberately calculating.
Cultural/Historical/Scientific/and Social Aspects
Significance of art form
pg. 17 During the field's long, half-forgotten early development, it centered around the “color
organ,” a device which embodied the light-as-music metaphor that continues to influence the
field today. The first color organ was designed in 1743 by Louis Bertrand Castel, a
mathematician and a priest.. associated the seven notes of the major scale to seven different
colors, which were displayed by colored papers moving in front of candles.

pg. 11 ...it was in the 1960's, first in America and then elsewhere, that the light-show became a
staple of rock gigs and hallucinatory 'happenings'. Before computers... the creation of light-shows
was cumbersome and haphazard. It involved bulky projectors, and usually lots of them. Oil was
poured on to glass and mixed with water; chemicals were exposed on photographic slides; and
hand-painted glass plates were subjected to multiple exposure and were projected through
spinning wheels. The results were kaleidoscopic, hallucinatory and sensational.
pg. 19 ...Seymour Locks, an art professor in San Francisco, gave improvised visual
accompaniment to a jazz combo by using an overhead to project wet paints that he swirled
around in a glass dish. This was the beginning of a dynamic, abstract- expressionist technique
that came to be known latter as “wet shows” or “liquids”
pg. 21 A few light artists, used live and recorded video during the late 1960's. In 1969 Nam
June Paik began experimenting artistically with the Sony Portapak, a mobile video unit developed
for television reporters covering the Vietnam War. But for the most part, the technology was too
expensive to catch on widely.
pg. 22 Another technical mini-revolution happened in 1984, when Australia's Fairlight
Corporation came out with the Fairlight CVI (Computer Video Instrument)...Fairlight operators
could superimpose graphics, produce solarizations and other color effects, show “trails,” and do
lots of other things that you see an awful lot of in videos of the mid-1980s...the introduction of
the Fairlight opened a brief window of opportunity during which designers could satisfy clients by
simply pressing the right button on a new tool, before overuse rendered the results recognizable
and uninteresting.
Roles of Vjs
pg. 10 Vjs now think of themselves as filmmakers, or perhaps moving-image alchemists, mixing
live visuals in front of real people; mixing the signs, symbols and imagery of the world, and re-
envisioning it for a jaded postmodern culture that claims to have seen everything. Whether this
alchemical process is done in a club as an adjunct to the spinning turntables of a DJ or as is
increasingly likely, in concert venues where the visuals accompany a live music performance, the
role of the VJ is to creature audio-visual marriages that engage the senses.

Blending of VJ and Interactive Art traditions

pg. 35 Kathleen Forde is an independent curator based in New York who has developed new-
media exhibitions and performance series for institutions and festivals Including SFMOMOA, and
the Philadelphia Museum of Art...

pg. 39 So to me, something like Jitter is a profound artistic movement, in terms of both what the
software is enabling the artistic community to do, and also the way that it's going about it – the
way it's a community, like an open-source movement...And often the people creating the
software are also using it. So, perhaps the people creating the software, the programmers, will
be seen as the meta-artists of our time, not unlike the artists who created video imaging
equipment in the '70s.

Logical progression utilizing the newest technology

pg. 24 ...others wrote software to make it all easier. The British DJ/VJ duo Coldcut collaborated
with Camart to build a video-triggering application called Vjamm, which they began performing
with in 1997 and distributed to the public in 1999... Over the next few years, Arkaos and Vjamm
were joined by scores of other VJ applications running on all major platforms.

Artistic Aspects
Improvement of storytelling
Usage of tactile / organic interface
pg. 66 (interview c505) People will continue to explore experimental visuals using tools like
Jitter, but they'll tire of the lack of control these tools offer and begin to move toward a less real
time but more controllable approach.
We would like to try to integrate some live instrumentation in our work in the audio realm.. We're
interested in taking the performance beyond the laptop somehow.

Relate to Collage / Gesture Field Painting

Pg 156 This association between physicality and expressiveness, implicit in gesture- field
painting and spontaneous collage, would be developed further in media that intrinsically engaged
the body to an even greater degree: specially, clay pottery and modern dance.

pg. 134 ...Robert Motherwell and Lee Krasner developed collage into an art form equal in
significance to Jackson Pollock's gesture-field painting. The development of abstract
expressionism in the late 1950's toward assemblage, combines, and happenings owed much to
these pioneers of collage.

pg. 135 Motherwell was content to let the pasted paper be the primary element in his work. He
emphasized the tactile qualities of the paper, not its use in figural representation. As he
explained in 1946,
The sensation of physically operating on the world is very strong in the medium of the papier
colle' or collage, in which various kinds of paper and pasted to the canvas. One cuts and
chooses and shifts and pastes, and sometimes tears off and begins again. In any case, shaping
and arranging such a relational structure obliterates the need, and often the awareness o of
representation. Without reference to likenesses, it possess feeling because all the decisions in
regard to it are ultimately made on the grounds of feelings.

Programming sketches as new form of mix tape

pg. 23 ...solitary artists edited video at home, producing dense, complex “scratch videos” -
collage compositions which often carried political or social critiques. These were traded around
on tapes and shown in basements and clubs. Meanwhile... more social video geeks pursued live
Vjing, using simpler tools to play with video or generate abstract patterns on the fly, without
sound, in order to accompany music from the DJ.

Technical Aspects
Description of Development plan
Description of setup / usage / physical aspects of system
pg. 40 The business of concert visuals and visual design is expanding alongside the release of
newer more affordable playback technologies and software applications. There are dozens of
media servers on the market, increasingly powerful and lower in cost than previous generations.
They are capable of playing back multiple layers of video, combined with digital effects and color
correction, and can be controlled by lighting consoles or MIDI devices, opening the door for a
wide range of performance methods.

Integration of existing components / libraries

pg. 511 Messa di Voce is an audiovisual performance and installation in which the
speech, shots, and songs produced by two vocalists are augmented in real time by custom
interactive visualization software. The project touches on themes of abstract communication,
synesthetic relationships, cartoon language, and writing and scoring systems, within the context
of a sophisticated and playful virtual world...
Messa di Voce lies at an intersection of human and technological performance extremes,
melding the unpredictable spontaneity of the unconstrained human voice with the latest in
computer vision and speech analysis technologies.
pg. 548 Messa di Voce... uses whole-body vision interactions similar to Krueger's, but combines
them with speech analysis and situates them within a kind of projection- based augmented
reality. In this audiovisual performance, the speech, shouts, and songs produced by two abstract
vocalists are visualized and augmented in real time by synthetic graphics, To accomplish this, a
computer uses a set of vision algorithms to track the locations of the performer's heads; this
computer also analyzes the audio signals coming from the performer's microphones. In response,
the system displays various kinds of visualizations on a protection screen located just behind the
performers; spoken and sung. With the help of the head-tracking system, moreover, these
visualization are projected such that they appear to emerge directly from the performer's mouths.
Capabilities of these for Tracking / Audio / Visualization
pg. 100 (interview ds-x.org) We're developing patches for our work that are mostly specialized
in one direction – based on the idea of the project... We are using the following software: Max/
MSP Jitter and SoftVNS, SuperCollider, EyeCon and Kalypso... For dance productions we are
currently using different motion-tracking systems. EyeCon from Frieder Weiss gives us a good
instrument to analyze the performance space. This specialized software gives us the perfect
opportunity to connect these analyzed parameters to visual and acoustic systems like Max/MSP
Jitter and SuperCollider. Another piece of software, also from Frieder Weiss is Kalypso. It's a
motion-tracking system like EyeCon, but more specialized for real-time visuals. The camera input
is the only source, and its picture is transformed and manipulated inside Kalypso. The output is
the result of this manipulation process. So, the content material is on the one hand the real-time
picture, and on the other prepared and precaptured material that was filmed and edited... Sound
is the basis of our VJ work. The sound controls – directly or indirectly – the parameters of the
resulting picture, the colors, size or position. The music style is relatively unimportant. We use
the music's basic parameters like beat, frequency, or volume to control the picture.

Conclusion
Design aimed toward goal of Improvisation / Expression
pg. 191 It's improvised and performed in front of a live and generally demonstrative audience,
which makes it very different from movies or television. It has traditionally been in the
background, ambient, but in new settings, it is coming more to the fore. It can be abstract,
representational, textual, symbolic, or any combination. All of these factors define how the form
fits in with other media, and predict that in goods hands, its cultural influence could be
enormous.

pg. 193 The hottest environments of all are live performances in venues small enough that the
audience can be actively engaged. Vaudeville, improvisational comedy, and some forms of
experimental theater fit into this category. Given the infinite breadth and improvisational power if
Vjing, which lets performers show virtually anything they want and change it immediately, this
also seems like the logical category for VJ performances, when they're center-stage in front of
an attentive audience.
pg. 194 Improv comedy itself is a well-developed field with plenty of wisdom for VH, including
how to successfully improvise narratives. Most of these techniques are based on interactive role-
playing, however which won't be relevant to Vjs until they start performing as teams within multi-
player virtual worlds.

Implications community based sharing of sketches / other performers

pg. 190 To encourage the conversational dynamic, bebop not only expanded the expressive
range of each instrument, but also provided for each instrument to have its own register in the
overall arrangement of sounds...With its conversation dynamics and emphasis on the cadences
of speech, bebop can be understood as a music in which the instruments are the “voices” of
musicians speaking through prosody alone.
Importance of Visual performance as shared experience
pg. 41 Vjs can look forward to concerts and live events incorporating significantly more visual
artistry over the next ten years. Technological innovations are not likely to slow down, and more
and more musical artists now depend largely on ticket sales from live concerts to drive their
careers, rather than just album sales. Today's audience wants to be entertained in more and
more exiting ways an concert visuals can do this. In this climate, the VJ director becomes an
incredibly valuable commodity in the entertainment market.

Bibliography

Web:

http://www.tmema.org/messa/

http://mtg.upf.es/reactable/

http://createdigitalmotion.com/2009/03/02/structuring-a-vj-set-with-vdmx-clip-prep-on-the-
 new-vj-kung-fu/

http://www.jazzmutant.com/lemur_overview.php

http://www.flong.com/projects/aves/

http://vjkungfu.com/archive/loop-r-vj-touchscreen-system/

http://www.reas.com/texts/sonicacts.html

Book:

Faulkner, Michael. VJ: Audio-Visual Art and VJ Culture. London: Laurence King Publishing Ltd,

2006

Spinrad, Paul. The VJ Book. Los Angeles: Feral House, 2005

Fry, Ben, and Casey Reas. Processing: a programming handbook for visual designers and

artists. Cambridge: The MIT Press, 2007

Arnheim, Rudolf. Art and Visual Perception. Berkeley: The University of California Press, 1974

Belgrad, Daniel. The Culture of Spontaneity. Chicago: The University of Chicago, 1998.