8/8/2018 David Rokeby : The Harmonics of Interaction

Rokeby : Home / Works / Current Shows / Texts / softVNS / Links / e­mail me

David Rokeby : The Harmonics of Interaction

(published in MUSICWORKS 46: Sound and Movement, Spring 1990)

In the series of installations that fall under the general title 'Very Nervous System', I use
video cameras, image processors, computers, synthesizers and a sound system to create a
space in which the movements of one's body create sound and/or music. Within the
installation, sound has a sculptural presence, both as an extension of the body, and as a
physical reality which one encounters with the body.

While the 'sound' of the system and the 'dance' of the person within the space are of interest
to me, the central aspect of the work is neither the 'sound' nor the 'dance'. It is the
relationship that develops between the sounding installation and the dancing person that is
the core of the work.

(In one piece there was a sound you could only find if you walked as though you were
carrying a 40 pound weight.)

The installation watches and sings; the person listens and dances. But the relationship that
develops is not simply that of a dialogue between person and system. Dialogue in its back­
and forthing implies a separation of the functions of perceiving and responding. But for the
installation, perception and expression are virtually simultaneous. As a result the installation
and participant form a feedback loop which is very tight, yet very complex...

The installation when empty stands as a silent awareness, a fragment of a nervous system:
cameras for eyes... computerized visual cortex... in software a complex set of filters and
translators (a set of opinions, a rudimentary aesthetic?)... synthesized vocal cords...
speakers for mouths...

With first movement of a body within the perceptual range of this system, this synthetic
awareness is engaged and begins making sound.

The sound propagates through the air, enters the ear (and body) and travels through the
transformations of human acoustic perception, the labyrinth of the individual's brain, and the
tangled interface (psychological and physiological) linking mind and body, to manifest itself
as further movement (or lack thereof) which completes a cycle and starts a new one...

But the cycles blur into one another because as soon as one moves in response to the
sound of the system, the system is already modifying its response to reflect those
movements and so on, so that body and system are drawn into constant readjustment and a
sort of oscillation begins.

The minimum time delay of this loop is about 1/30th of a second. But the real delay depends
on the exact path(s) through mind, body and installation. Both the system and the person
operate as prisms, splitting the 'beam' of feedback into different components, travelling at
different speeds, oscillating at different frequencies...

And so, just as acoustic feedback exaggerates aspects of the acoustic nature of a room, the
interactive feedback loop in my installations reinforces particular aspects of the system and
http://www.davidrokeby.com/harm.html 1/3
8/8/2018 David Rokeby : The Harmonics of Interaction

the person in it, producing resonances. These can be thought of as the harmonics of the
interaction and are unique for each different combination of installation and individual.
(These harmonics often come as a complete surprise to me; I have sometimes observed
people creating sounds and patterns in interaction with the installation which I, knowing the
program intimately, would have thought impossible.)

Because of the overall complexity of the system, there are many factors which determine its
resonant characteristics. These factors can be grouped as functions of the body, and the
character of the person in the space, and the sound, and the responsive behaviour of the
installation itself.

Each body has its own resonant frequencies which are functions of size, shape, weight and
musculature. Rhythms inherent in the interactive program interfere with and/or reinforce
these. (It is interesting to watch a large number of people on a dance floor, each striving to
find a way to bring their uniquely shaped body into resonance with the rhythms of the music.
Each different popular dance style seems to have evolved out of the need to configure the
body in such a way that it can resonate to a particular kind and rate of rhythm.)

The nature of the sounds that I use in the installation have a less direct, but no less powerful
impact on the interaction. People seem to endow a sound that has a lot of texture with an
external material reality, find themselves imagining the feel of it against their body, imagining
the space filled with sound particles. Rounder, deeper sounds seem to manifest themselves
more as a presence within or an extension of the body, something of the stomach, organs
and muscles rather than of the skin... The body tends to adopt different relationships to the
installation depending on whether the sounds are felt on the inside or the outside, and each
relationship has its own harmonics.

Within the system's harmonic spectrum, different ranges are brought into play as the
interactor responds with different degrees of mental reflection and spontaneous action. If
they adopt a predominantly intellectual approach to the installation, they tend to reject
immediate participation in the feedback, working more like a scientist, making experiments
and 'objectively' observing the results. The aim of such an approach is often to learn to
control the system in order to be able to play it like an instrument. In this sort of situation, the
system resonates, if at all, at very low frequencies, because the person shapes their
movements over time in response to a slowly accumulating body of experience. (The
intellect operating as a low pass filter.)

(The body tends to betray those who try to adopt a purely intellectual approach with the aim
of exerting control over the system. The first test that such people apply to the system is 'If I
make the same movement twice, will I get the same sound?' In some of my simpler
interactive pieces, one will get the same sound for exactly the same movement. Within such
an installation, a pattern emerges. A person makes a gesture, as though asking a question
of the space and notes the system's response. They make a second gesture and a third,
again as questions, and both times receive the same response as the first. At this point, they
pause, a change comes over their face as if to say 'Yes I understand I have control', and
they make a fourth gesture, no longer as a question, but as a statement, almost a command.
And they almost invariably get a different sound, the shift in attitude manifesting itself in a
subtle but noticeable shift in the carriage of the body and of the instant­to instant dynamics
of the gesture.)

If, on the other hand, they allow themselves to respond spontaneously to the music of the
system, it is they who are played by the installation. This approach involves opening oneself
to suggestion, allowing the music of the system to speak back through one's body directly,
involving a minimum of mental reflection, and thus tightening the feedback loop as much as
possible. This sort of situation reinforces the higher frequencies and dampens the lower
http://www.davidrokeby.com/harm.html 2/3
8/8/2018 David Rokeby : The Harmonics of Interaction

ones...

Among those who approach the installations in this manner I often observe people taking
responsibility for perceptual errors that the system's software makes, unconsciously
justifying these mistakes by deflecting their intended gesture into harmony with the system's
response; conversely, they will often attribute manifestations of their own intelligence to the
system. (Having unconsciously fallen into synch with a rhythm in the music, people often
express amazement at the system's ability to synchronize to their movements.)

Of course these are gross generalizations, and everyone who enters the system employs a
combination of each approach in different measure. My ideal scenario is one in which the
two approaches are balanced, resulting in a 'broad bandwidth' of interaction.

When I design the programs that define the reactive behaviour of the installation, I work
somewhat like an sound engineer at a amplified concert who uses filters and other distorting
devices to control the feedback of the sound system. However, my aims in doing so are
different. The engineer seeks to cancel the acoustic idiosyncrasies of the room and system
so that they don't distract from the performance, and to eliminate feedback. I filter and distort
in order to highlight the feedback, tuning it to reinforce the hidden physiological and
psychological resonances of the body and mind in physical interaction.

The music of the 'Very Nervous System' installations is not so much in the sounds that you
hear, but in the interplay of resonances that you feel as you experience the work with your
body.

Rokeby : Home / Works / Current Shows / Texts / softVNS / Links / e­mail me

Copyright 1996 David Rokeby / Very Nervous Systems / All rights reserved. 3/7/96

http://www.davidrokeby.com/harm.html 3/3