SEE THIS SOUND

Audiovisuology
A Reader

Edited by 
Dieter Daniels and Sandra Naumann
In today’s audiovisual media culture, the combination of images and sounds is a mat-
ter of course. The technical and aesthetic boundaries between the two have become
fluid. Digitization has broken down auditory and visual information into bits and
bytes that can be linked or translated at will. At the same time, a theoretical interest
in the concepts behind the combination of images and sounds has emerged, uniting
the arts and their history.

Since the end of the nineteenth century, with the rise of technical media, efforts
have been made to reproduce, intensify, expand, and finally to transfer audiovisual
perception to virtual worlds that promise to fulfill utopian fantasies of synthesis. In
the tension between naturally and artificially (or artistically) produced audiovisuality,
between its immersive or analytical implementation, the question arises to what ex-
tent we actually are conscious of what happens between the auditory and the visual
realms. The convergence of the audiovisual is not just a matter of technology—per-
ception research also increasingly calls the separation of the senses into question.

At the same time, there is no genuinely audiovisual discourse about hybrid audiovi-
sual artworks themselves: these are usually evaluated in specific context, localized,
for example, either in the visual arts, in music, in film, or in club culture. Critical
debate conforms to different standards in each case, while it also often ignores one
of both sides of the audiovisual construct. Even if media culture has become multi-
sensory, an established “in between” that could do justice to hybrid art forms does
not yet exist.

This new edition of See This Sound. Audiovisuology combines two volumes in a
single reader and can be seen as a sizable contribution to filling this gap:
The first volume, “Compendium,” bundles information from individual academic
disciplines and offers a comprehensive foundation of knowledge on the diverse
relationship between images and sounds.
The second volume, “Essays,” offers in-depth studies on the historical development
and the theoretical framework of our audiovisual society.

Even though an established discipline of audiovisuology does not (yet) exist, it takes
shape here as an intersection—or, better, a sum—of the thematic areas. This publica-
tion reveals audiovisual research to be one of the great fields of experiment in the
modern era—one which, like the project of modernism as a whole, has not been and
cannot be brought to a close.
Audiovisuology—new edition—two volumes in one reader!
Audiovisuology is not a new science, but the intersection of all existing research on audiovisual
art forms and related disciplines. This book is the first comprehensive publication on the relation
of sound and image in art, science, and technology.

Volume 1. Compendium: Volume 2 . Essays:

An Interdisciplinary Compendium Histories, and Theories of Audiovisual
of Audiovisual Culture Media and Art

The first volume is an extensive The second volume offers multiple

compendium of texts that explore
both the different art forms in which
image-sound relationships play a
significant role as well as the methods
that have been used to link acoustic and
visual phenomena. Thirty-five separate
articles describe the entire spectrum
of the audiovisual arts and of audiovisual
phenomena by means of longitudinal
sections of history and systematic
cross sections, while examples are
presented in abundantly illustrated
analyses of individual works.
perspectives on the hybrid cultural field
between image and sound. The essays
deal with such topics as the archeology
of audiovisual media between science
and art, mutual influences between
pop culture and the visual arts, the
hybridization of machines and emotions
in the process of artistic production, and
the search for intensitiy in audiovisual
environments with a focus on their bases
in the psychology of perception.

Authors
Claudia Albert, Amy Alexander, Rainer Bellenbaum, Hans Beller, Sabeth Buchmann,
Michel Chion, Dieter Daniels, Gerhard Daurer, Diedrich Diederichsen, Hinderk M.
Emrich, Barbara Flückiger, Golo Föllmer, Tina Frank, Maureen Furniss, Marc Glöde,
Andrea Gottdang, Florian Grond, Boris von Haken, Justin Hoffmann, Christian Höller,
David E. James, Jörg Jewanski, Barbara John, Henry Keazor, Barbara Kienscherf,
Constanze Klementz, Katja Kwastek, Jörg Lensing, Golan Levin, Lia, Cornelia Lund,
Helga de la Motte-Haber, Jan-Philipp Müller, Sandra Naumann, Janina Neufeld,
Chris Salter, Birgit Schneider, Julia H. Schröder, Theresa Schubert-Minski,
Marcel Schwierin, Simon Shaw-Miller, Jan Thoben

Edited by Dieter Daniels and Sandra Naumann with Jan Thoben

9 783863 356132

Verlag der Buchhandung

Walther König, Köln
ISBN 978-3-86335-613-2 www.see-this-sound.at
Volume 1. Compendium
An Interdisciplinary Survey of Audiovisual Culture

Themes
Abstract Film / Animation / Architecture / Artist-Musicians / Cinedance / Color
Organs / Dance / Expanded Cinema / Film Score / Games / Gesamtkunstwerk /
Graphic Notation / Interactive Art / Light Shows / Literature / Live Visuals / Music /
Music Video / Musical Theater / Painting / Performance Art / Software Art /
Sonification / Sound Art / Sound Design / Video
Audiovisual Perception / Color-Tone Analogies / Conceptual Correlations /
Montage / Parameter Mapping / Structural Analogies / Synchronization /
Synesthesia / Transformation

Volume 2 . Essays
Histories and Theories of Audiovisual Media and Art

–H  ybrids of Art, Science, Technology, Perception, Entertainment, and Commerce

at the Interface of Sound and Vision
– S eparation and Conjunction: Music and Art, circa 1800–2010
– D eaf Dumb Mute Blind: Artistic Approaches to Image/Sound Relationships in
Pop Culture
– The Expanded Image: On the Musicalization of the Visual Arts in the Twentieth
Century
– B etween the Poles of Mickey Mousing and Counterpoint
– S ound and Image Worlds in Pop Music
– Audiovisual Interactive Art: From the Artwork to the Device and Back
– O n Hearing Eyes and Seeing Ears: A Media Aesthetics of Relationships
between Sound and Image
– The Question of Thresholds: Immersion, Absorption, and Dissolution in the
Environments of Audio-Vision
– Audition and Ergo-Audition: Then and Now
4

Compendium

Dieter Daniels, Sandra Naumann, Introduction Vol. 1, in: Dieter Daniels, Sandra Naumann (eds.),
Audiovisuology, A Reader, Vol. 1: Compendium, Vol. 2: Essays, Verlag Walther König, Köln 2015,
pp. 5 - 16
 5

Introduction
Dieter Daniels and Sandra Naumann

The Perennial and the Topical

The time span and thematic area covered by this compendium range respec-
tively from antiquity to the present day and from philosophical models to exist-
ing apparatus and tools, for example from the Pythagorean theory of universal
harmony to the audiovisual software of music visualization plug-ins for mp3
players. The immense field under examination, both with respect to its content
and its history, is, however, held together by an abundance of cross-references.
This introduction will examine how apparently perennial, supra-historical ques-
tions are intertwined with highly topical issues. The relationship between his-
tory and the present is at times almost paradoxical here. Scientific and techno-
logical progress may lead one to believe that a wealth of conundrums and false
turns from antiquity to the eighteenth century have been definitively resolved,
whereas in actual fact questions that have been debated for hundreds and
thousands of years are still being reformulated in similar ways today. The differ-
ences in methodology and chronology found across the many arts and sci-
ences concerned complicate the issue further. This compendium seeks to
address the problem by examining the subject matter from multiple specialist
perspectives and by approaching it from two different angles: both historically
(in section I) and systematically (in section II).

The point of departure is a review of the artistic fields and forms in which the
current multiplicity of relations between sight and sound manifests itself. 1

In today’s media-oriented society, the coupling of images and sounds has

become as ubiquitous as it is inescapable. Through audiovisual technology, not
only hearing and seeing, but also the aesthetics, technology, and economy of
the visual and the auditory have become connected with one another in multi-
faceted ways. This applies equally to our leisure activities and our work envi-
ronments, to the active production of audiovisual content, and to the reception
of the mass media. As indicated by the neologism “prosumer,” production and
consumption can no longer be sharply distinguished. Embedded as we are
today in an audiovisual media environment, we find it difficult to imagine a time
when the technical ubiquity of images and sounds did not exist. Yet we need

1 The analysis of the topic’s history, proceeding from the present day, is the leitmotif of the
entire See this Sound project, including the associated exhibition. Cf. Dieter Daniels and Stella
Rollig, preface to See This Sound. Promises in Sound and Vision, Lentos Kunstmuseum Linz,
eds. Cosima Rainer, Stella Rollig, Dieter Daniels, and Manuela Ammer (Cologne: Verlag der
Buchhandlung Walther König, 2009), 12.
6

only look back barely one hundred years to realize that for the longest period in
the history of civilization, the auditory and the visual were not technically linked.

The history of sound-image correlations began long before the media age, as
the reader will learn from many of the contributions to this compendium. Three
distinct pre-histories, which are fundamentally separate but also selectively
connected, emerge:

1 The theory and practice of relationships between colors and sounds.

Over history, numerous mythological, philosophical, mathematical, physical,
and metaphysical models were constructed that postulated the correspon-
dence of colors and sounds. 2 These models were often encyclopedic sys-
tems of analogies and references between planets, metals, the cardinal
points, seasons, numbers, flora, and fauna; they were an expression of the
yearning for a holistic formula to explain the world, which would subsume
the cosmos and the psyche under the harmony of a higher order. They were
also the point of departure for repeated experiments to construct color
organs designed to translate such theoretical models into perceptible evi-
dence, but these ultimately foundered on the reality that an intersubjective
standardization of color-sound correspondences simply is not possible. 3
Although such models have been outmoded by modern physics research
and media technology, they nonetheless dealt with questions that remain
relevant today both for the study of neurological synesthesia and for the
sonification and digital parameter mapping of visual and acoustic data. 4

2 T
 he evolution of human perception.
This concerns the differentiation and (re-)synthesis of hearing and seeing
over the course of natural evolution and their subsequent cultural condi-
tioning, an aspect of human evolution that is represented by multimodal
integration as an element of the perceptual capacity of the individual. 5 Sev-
eral anthropological theories dating from the early twentieth century are
based on the assumption that the senses had a single common precursor
from which the individual sense faculties developed over the course of evo-
lution. Also, it is allegedly possible to demonstrate that certain “primeval
synesthesias” existed over the course of human development and history. 6
Today, neurologists are exploring the hypothesis that during early neonatal
development the sensory regions in the brain advance from synesthetic
processing to neurologically differentiated, single-sense processing. 7

3 T
 he combination of auditory and visual forms of expression in
human culture.
Since human prehistory, live performances of rituals and artworks have
combined sight and sound as articulated by the body, voice, gestures, and

2 See the chapter “Color-Tone Analogies” by Jörg Jewanski in this volume.

3 See the chapter “Color Organs“ by Jörg Jewanski in this volume and the comparison of dif-
ferent analogies in terms of the position of the color red in “Color-Tone Analogies” by Jörg
Jewanski in this volume.
4 See the chapter “Synesthesia” by Hinderk M. Emrich, Janina Neufeld, and Christopher Sinke,
the chapter “Sonification” by Florian Grond and Theresa Schubert-Minski, and the chapter
“Parameter Mapping” by Tina Frank and Lia in this volume.
5 See the chapter “Audiovisual Perception” by Gerhard Daurer in this volume.
6 On primeval synesthesia, see Albert Wellek, “Die Farbe-Ton-Forschung und ihr erster Kon-
greß,” Zeitschrift für Musikwissenschaft 9 (1927): 576–584.
7 See Daphne Maurer, “Neonatal Synaesthesia. Implications for the processing of speech and
faces,” in Synaesthesia. Classic and Contemporary Readings, eds. Simon Baron-Cohen and
John E. Harrison (Oxford: Blackwell, 1997), 224. Also see the chapter “Synesthesia” by Hin-
derk M. Emrich, Janina Neufeld, and Christopher Sinke in this volume.
Introduction 7

mimicry in dance, theater, and music, and with the support of costumes,
masks, and musical instruments, not to mention the use of light. From
sacred torch-lit dances in prehistoric caves to the sound of the organ under
the stained-glass windows of cathedrals, the creation of an audiovisual
whole was deemed to be an extraordinary experience, often with a spiritual
meaning. Whereas these audiovisual expressions were bound to the
moment of their execution, technical mass media have now enabled the
conservation and reproduction at will of auditory and visual sensory
impressions on film, video, and DVD. Nonetheless, live performance is cur-
rently experiencing a renaissance: especially in the live visuals found in club
culture, the transcendence and corporeal immediacy of an audiovisual cou-
pling on the basis of the new media are celebrated more excessively than
ever before.

At all three levels, then, nature, culture, and technology overlap: the physical,
physiological, and perceptual basic conditions, and their active and conscious
human shaping through cultural and artistic practices as well as their potential
expansion by means of technical media. 8 All three levels also show how the
apparently perennial and highly topical aspects are intertwined with one another.

But for all this entanglement between history and the present, the state of
knowledge is different today, while the questions we pose are also different to
those of our predecessors for there have been three decisive changes in the
boundary conditions:

— Since the development of modern physics and the work of Isaac Newton and
Thomas Young, we know that light and sound are two entirely separate phe-
nomena: sound waves are oscillations of pressure that travel through a gas, liq-
uid, or solid, which is why outer space is silent (there is no air), while what we
refer to as light is that small part of the electromagnetic spectrum visible to the
human eye—the same spectrum that comprises both the microwaves of kitchen
appliances and the long waves of radio bands. Physically, it is not possible to
integrate the frequencies of light and sound in an overarching cosmic harmony,
as has been attempted time and again since the days of Pythagoras, and nei-
ther can they be placed in a mathematically expressible proportional relation
with the planets, metals, cardinal points, seasons, and so forth.

— We know that our sensory perception is the actual “location” where light and
sonic waves meet to become audiovisual experience because we have eyes and
ears and our brain processes and interrelates the signals from theses sense
organs in parallel. The complexity of our system of sensory perception has
tasked a number of scientific disciplines from Hermann von Helmholtz’s physi-
ology up to contemporary neurological synesthesia and intermodal research,
but is still imperfectly understood.

— Since the era of Thomas Edison, we have been constructing many and
diverse audiovisual media devices, which in the meantime have become an
integral part of our lives. In the nineteenth century, the first cinematographs
and phonographs occasioned amazement and even fright; today, by contrast,
personal privacy and public spaces are invaded by pervasive and often aggres-
sive audiovisual messages or by omnipresent “ambient media” that are per-
ceived almost below the threshold of consciousness.

8 On the difference between automatic “multimodal integration” and the conscious and active
creation of “intermodal analogies,” see the chapter “Audiovisual Perception” by Gerhard
Daurer in this volume. A model with five levels is developed by Michael Haverkamp.
8

Today we are aware that our synthesis of hearing and seeing is a complex, sub-
jective achievement, which has no counterpart in the physical nature of light
and sound. Thus, the centuries-old quest to discover an analogy in nature
between optical and acoustic phenomena failed of necessity. However, we have
created in audiovisual media a counterpart to our complex achievement of syn-
thesis, which we take entirely as a matter of course and which surrounds us like
a “second nature.” The yearning for correspondences between sounds and
images has been satisfied by a techno-cultural achievement, and not by specu-
lation about the physico-mathematical structures of the optical and the acous-
tic. Nonetheless, the mathematical models of correspondence that have been
developed since classical antiquity are more relevant today than ever before
because digital technology has rendered the optical and the acoustic de facto
calculable, transformable, and manipulable at will. 9

A Possible and Impossible Chronology

The paradox inherent in the subject area (that it is at once perennial and topi-
cal) is also reflected in this compendium of audiovisuology. Compiling an over-
all timeline from the chronological depictions of individual art forms proves to
be difficult. The time frames of the individual themes vary too much: the con-
nections between painting and architecture, on the one hand, and music, on
the other, go back to classical antiquity; the audiovisual techniques of synchro-
nization date from the late nineteenth century; audiovisual software, sound
design, and live visuals developed only in recent decades with the aid of digital
technology. 10 At the same time, all of the academic disciplines involved have
meaningful timelines for their own particular subject, and there are also a num-
ber of common historical reference points that are considered as being of key
significance, albeit in each case for different reasons.

In this way, a web of parallel narratives develops that has interlinks and
stretches of common history with attendant bifurcations, but possesses no uni-
versal model in which each of the art forms, media technologies, and media
practices dealt with here has its explicit, historical, and systematically defined
place. The biased perspective of academic disciplines is demonstrated in an
exemplary way by the manner in which the auditory is separated from the
visual. The “deafness” of the disciplines that engage with images, and the
“blindness” of the disciplines that engage with music and sound are of seminal
relevance to the central concern of this volume. The attempt to delineate the
transdisciplinary field of audiovisuology encounters similar problems to those
described by Bruno Latour for science studies: the socio-technical networks
that exist between the individual disciplines are not visible from the perspec-
tive of the disciplines themselves; at the same time, they have real effects that
constantly defy scientific explanation. 11 The main concern of this volume is to
take the interconnection points and synergy effects of the different disciplinary
perspectives and to render them useful within a network comprised of the the-
ories and disciplines involved.

The study is also complicated by the fact that the speed of audiovisual praxis
today far outstrips that of theory formation. In the areas that are currently most

9 See the chapter “Parameter Mapping” by Tina Frank and Lia in this volume.
10 See the chapters “Painting” by Andrea Gottdang, “Architecture” by Ulrich Winko, “Synchroni-
zation” by Jan Philip Müller, “Software Art” by Golan Levin, “Sound Design” by Barbara Flück-
iger, and “Live Visuals” by Amy Alexander, all in this volume.
11 Bruno Latour, We Have Never Been Modern, trans. Catherine Porter (Cambridge, MA: Harvard
University Press, 1993), 3–11.
Introduction 9

active, the process of defining historical situatedness and theoretical contextu-

alization only begins retrospectively and at the same time serves as a strategy
to legitimize the establishment of new forms of art. This is the reason why artis-
tic self-contextualization often leads to historiography about precursors a pos-
teriori, as in the example of the ocular harpsichord of Father Louis-Bertrand
Castel from the eighteenth century, which is frequently claimed both as the
forerunner of VJing as well as of audiovisual games and music videos. Particu-
larly these very new fields often develop a desire for a historical pedigree, both
with regard to their own ancestry as well as to the aforementioned genealogi-
cal research. Some genres, such as music video, for example, have in the mean-
time acquired the status of independent art forms; others, such as abstract film
and abstract painting, are established art forms operating at the fringes. Some
of these fields do have relatively clear time frames; for example, abstract paint-
ing as visualized music from around 1900 to 1920, or Absolute Film from 1920
to 1930. 12 In addition, there are thematic fields that defy classification in any of
the established disciplines, forever lying “in between,” and that for this reason
have thus far been seriously neglected—the two-hundred-year-old history of
color organs and the much longer history of color-sound analogies, which color
organs sought to depict, are examples of this. 13 Scholarly research on these
subjects risks treating them as relatively hermetic, specialist fields, as though
they followed their own unique historic genealogy and an intrinsic logic that
can only be explained in the context of their history. Yet it is especially this
ambivalence and the negotiation of the position of such phenomena in the in-
between that makes these topics highly interesting.

From this complex structure of mutually overlapping systems, extensive affini-

ties, and mutual exclusions among the respective specialist narratives, two
models can be extrapolated in the search for an overarching chronology. The
first is a linear history of progress, which is oriented on the actual feasibility of
the audiovisual and the technology that in the last approximately 150 years has
brought forth the modern media-oriented society. The second model is a his-
tory of perennial ideas, whose origins reach far back into the ancient world;
however, because these themes experience a revival in topicality from time to
time, this leads to the constant recurrence of certain motifs, sometimes as con-
scious resumption and sometimes as naive reinvention. This permanent updat-
ing of the history of ideas is often driven by technical innovations of feasibility.
To see this in terms of a one-sided cause-and effect-schema, though, is inade-
quate because time and again elements from the history of ideas stimulate the
search for what is technologically feasible. This was already the case with Cas-
tel’s ocular harpsichord mentioned above, which its inventor initially presented
merely as a thought experiment. Through the debates it triggered, Castel found
himself obliged to deliver the empirical proof by constructing such a device, an
endeavor which was doomed to failure in view of the technology available at
that time. Thus, in this case there is no right or wrong model of a chronology;
both manners of representation have their specific justification. The result of this
dichotomy is the difficulty of compiling a comprehensive, overall presentation
that does justice to all aspects. Before this question is brought to a conclusion,
however, it will be useful to sketch the two possible models of a chronology.

Perception and Apparatus: A History of Progress?

The history of audiovisual technology can be represented in a relatively clear

12 See the chapters “Abstract Film” by Sandra Naumann and Marcel Schwierin and “Painting” by
Andrea Gottdang, both in this volume.
13 See the chapter “Color Organs” by Jörg Jewanski in this volume.
10

chronological order. Its far-reaching effects on the modern audiovisual environ-

ment are the main focus of the multidisciplinary perspectives of this volume.
Since the advent of telephone, phonograph, and film at the end of the nine-
teenth century, and since radio, sound film, television, audiotape, and video in
the twentieth century, audiovisual culture has undergone historically unparal-
leled expansion and reformation. All these media have redrawn the borders of
the visual and the auditory and reconfigured their relations. In the beginning, in
the nineteenth century, media first separated images and sound, then in the
twentieth century united them again. This led to the development of a new
diversity of machine-based artificial image and sound relationships. To cite just
one example: the synchronization of films is the technical affirmation of Michel
Chion’s synchresis, which he defines as the natural psychological automatism
of a motivic connection of simultaneous sounds and images.14 The gaps remain-
ing in these image-sound techniques have since become the area of activity of
(media) artists, who deconstruct their apparent naturalism and recombine its
elements so as to interrogate perception and medium on an ongoing basis.

Since the mid-eighteenth century, color organs have represented a kind of pre-
history of audiovisual media. There were numerous models of these apparatus;
some existed only as concepts, some were also actually constructed, and in
each case they were heatedly debated. 15 One could also describe these models
as pre-electronic media dream machines because they often sought to achieve
more than was actually possible with the technology of the period; nonethe-
less, they anticipated image-sound effects that later emerged as experimental
or innovative uses of audiovisual media.

Attempts to overcome the separation of image and sound using the media
machines of the nineteenth century (photograph, film, phototelegraph, tele-
phone, phonograph, gramophone) led to not very successful mixed forms such
as the Kinetoscope or Kinetophone. 16 The synchrony of these media combina-
tions of the mechanical, chemical, and electronic was constrained by clear limi-
tations. It was not until the 1920s that a significant step in the development of
audiovisual media was taken with the electrical processing of signals in optical
sound. Here, the sound is recorded using a microphone and optically recorded
as an oscillographic track on the edge of the filmstrip. Thus, for the first time,
both images and sound are recorded on the same storage medium. The optical
soundtrack is read with the aid of a photocell during its rendition and made
audible via loudspeakers. 17 “An eleven-fold transformation is necessary for the
complete metamorphosis, it is claimed,” wrote Siegfried Kracauer of this pro-
cess, adding that thus “the esotericism of technology today already surpasses
that of the Eleusinian Mysteries.”18 The most important achievement of the opti-
cal form was the precise synchronization of feature films with language and
music—resulting in the so-called talkies. And this had effects of greater import
than the mere addition of sound: it led to fundamental changes in the aesthet-
ics, methods of production, and economics of cinema films. 19

14 Cf. Michel Chion, Audio-Vision. Sound on Screen (New York: Columbia University Press, 1994),
63–64.
15 See the chapter “Color Organs” by Jörg Jewanski in this volume.
16 See the chapter “Synchronization” by Jan Philip Müller in this volume.
17 See the chapters “Transformation” by Jan Thoben and “Synchronization” by Jan Philip Müller,
both in this volume.
18 Siegfried Kracauer, 1928, reviewing the first sound films in Siegfried Kracauer, Der verbotene
Blick. Beobachtungen, Analysen, Kritiken, ed. Johanna Rosenberg (Leipzig: Reclam, 1992),
299.
19 See the chapters “Montage” by Hans Beller and Jörg Lensing, “Animation” by Maureen
Furniss, “Film Score” by Helga de la Motte-Haber, and “Sound Design” by Barbara Flückiger,
all in this volume.
Introduction 11

Furthermore, optical sound facilitated for the first time direct inter-transforma-
tion of acoustic and optical signals. This is a technological necessity, albeit not
the primary goal but a side effect of the work on synchronization. It inspired
artists such as Oskar Fischinger and engineers such as Rudolf Pfenninger to
explore the soundtrack as a creative medium. The far-reaching ideas only
yielded a few isolated experimental results because of the complexity of the
techniques involved. 20 The actual breakthrough to the universal formability of
the audiovisual did not occur until the 1960s with analog electronics, and in the
1980s with digital technology.

Digital technology’s development was highly diversified, which resulted in

complex possibilities for coupling and transforming audiovisual data that far
exceeded the “esotericism of technology” proclaimed by Kracauer in relation
to optical sound. For this reason, the development of digital technology is
explored in seven separate chapters in this volume. 21

Electronic modulation of image-sound signals has repercussions for all existing

audiovisual media that contain electronic components. Through digitalization,
electronics integrate all current media formats. All the devices that once led
separate lives in photography, film, video, radio, television, and audiotape now
run as emulations in the universal machine of the computer, so that audiovisual-
ity does not have to be generated by the combination of separate media, but is
implicitly and explicitly already given.

To give a preliminary résumé of this history of technological progress: in the

1920s, it became possible to represent images and sound as analog, electrical
oscillations; from the 1960s as audio-video signals; and from the 1980s as digi-
tal code in one and the same medium; with these innovations it also became
possible to inter-transform, generate, and manipulate images and sound. This
fact may now sound self-evident, but against the background of the long pre-
history, its importance cannot be overestimated. Before the advent of techno-
logical media, human perception was the only place where sound and light
came together. The centuries-old search for correspondences of images and
sounds, which derived from the experience of human perception, was doomed
to fail as an “anthropomorphism” for as long as it referred to the reality of these
physically completely separate phenomena. It is only through audiovisual
media that human perception has obtained a counterpart in the world of
machines—the audiovisual is now located both in the human senses and in
things.

A parallel might be drawn here between the history of technology and the bio-
logical and anthropological evolution outlined above: the increasing differenti-
ation of the sensory organs to the point where acoustic, visual, haptic, and
olfactory stimuli are separated is in a sense reversed in the history of media.
The initially separate acoustic and visual phenomena are increasingly merged
by technological progress. It is only in this way that the potential of audiovisual

20 Thomas Y. Levin, “‘Töne aus dem Nichts.’ Rudolf Pfenninger und die Archäologie des synthe-
tischen Tons,” in Zwischen Rauschen und Offenbarung. Zur Kultur- und Mediengeschichte der
Stimme, eds. Friedrich Kittler, Thomas Macho, and Sigrid Weigel (Berlin: Akademie Verlag,
2002), 313–355. English version Thomas Y. Levin, “‘Tones from out of Nowhere’: Rudolf Pfen-
ninger and the Archaeology of Synthetic Sound,” Grey Room 12 (Fall 2003): 32–79; available
online at www.centerforvisualmusic.org/LevinPfen.pdf (all Internet references in this volume
last accessed on November 30, 2009).
21 See the chapters “Video” by Yvonne Spielmann, “Transformation” by Jan Thoben, “Software
Art” by Golan Levin, “Parameter Mapping” by Tina Frank and Lia, “Interactive Art” by Katja
Kwastek, “Sonification” by Florian Grond and Theresa Schubert-Minski, and “Live Visuals” by
Amy Alexander, all in this volume.
12

technology approaches the primeval synesthesias of the human senses postu-

lated by anthropologists and color and sound researchers in the 1920s.

The centuries-old dream of “eye-music,” for which synesthesia has often been
used as a metaphor, has thus mainly arrived in the reality of appliances since
the rise of electronics. Without human associations or artistic interpretations
having to be involved, it is possible to generate images and sounds automati-
cally from the same signal, and to transform them into one or the other. 22 As in
the case of optical sound, the means to transform images and sounds was not
the goal of electronic media technology, which was actually designed for
audiovisual production and reproduction. But from this basic technical princi-
ple a creative spin-off and artistically innovative use of electronics developed
with its own, fascinating history. This ranges from the use of the oscilloscope
for visual music in the 1950s films of Mary Ellen Bute, Hy Hirsh, and Norman
McLaren, to Nam June Paik’s TV experiments of the 1960s (in which he fed the
audio signal of an audiotape into the cathode-ray tube of a television set), and
then onward to an entire generation of artist-inventors, who in the 1960s and
1970s worked with audio and video synthesizers on special effects and manipu-
lation techniques. 23 Finally, in the 1990s, digital signal processing enabled the
mapping of images onto sound or sound onto images, as well as their simulta-
neous generation according to the same parameters. This created precisely
what Golan Levin describes as “inexhaustible, infinitely variable, time-based,
audiovisual ‘substance’” that can be manipulated in real time. 24 In contrast to
the mainstream history of technological progress, these artistic and experimen-
tal applications link back to the long history of ideas of visual music. Such cre-
ative use of electronics for purposes other than those intended thwarts their
actual industrial and commercial functionality and the ostensible naturalism of
audiovisual high definition.

The artistically motivated image and sound experiments in visual music during
the 1920s, in intermedia art during the 1960s, and in media art during the 1980s
have entered the hybrid culture of digital mass media as standard procedures.
The now fluid technical boundary between image and sound has far-reaching
effects on all established genres (e.g., image-sound montage in cinema films
and television, live concerts with visuals, audiovisual ambience, and art installa-
tions). Its subliminal efficiency often has more significant consequences than
are demonstrated manifestly in a direct image-sound transformation. The
hybridization of the technical basis of all audiovisual media is of fundamental
importance both aesthetically and economically. Because there is no longer
any differentiation between the channels of distribution, models of marketing,
and output media of sound and vision, the synthesis of the arts that the avant-
garde movements of the nineteenth and twentieth centuries called for is no
longer a question of technical feasibility. Instead, today the artistic genres are
separated again more distinctly at the cultural surface than was envisaged by
the new spirit of optimism surrounding visual music in the 1920s or in the inter-
media euphoria of the 1960s. The theories of intermedia art and the Gesamt-
datenwerk (integrated data work) may be technically realizable through digita-
lization, but they forfeit their character of a cultural utopia. 25 Unlike the

22 On the different approaches and processes to connect visual and auditory arts or phenom-
ena, see the second section of this volume with its chapters “Conceptual Correlations” by
Sabeth Buchmann and Rainer Bellenbaum, “Montage” by Hans Beller and Jörg Lensing,
“Parameter Mapping” by Tina Frank and Lia, “Color-Tone Analogies” by Jörg Jewanski, “Syn-
chronization” by Jan Philip Müller, and “Transformation” by Jan Thoben.
23 See the chapter “Video” by Yvonne Spielmann in this volume.
24 See the chapters “Software Art” by Golan Levin and “Interactive Art” by Katja Kwastek, both
in this volume.
25 See the chapter “Gesamtkunstwerk” by Barbara John in this volume.
Introduction 13

Gesamtkunstwerk (total art work) of the nineteenth century, today praxis is

forging far ahead of theory. Accordingly, the audio and visual arena is situated
less in high culture, which in many places is again defending the specificity of
its genres and focusing them aesthetically, and more in mass-media-permeated
everyday culture and the perceptional habits informed by it. The utopia and
praxis of a programmatic, theoretical, and aesthetic emphasis has been
replaced by the permanent linkage of image and sound as a commodity, which
proves itself to be, for good or for worse, more of a way of life than an art form.

Eternal Recurrence—or Constant Reinterpretation?

A linear chronology of the evolution of relations between image and sound as
outlined above is very straightforward from the perspective of media technol-
ogy, especially, but still neglects important aspects of the historical multiper-
spectivity of the subject. In spite of this deficit, in many overview publications it
is this history of progress that has become established as the way to present
the subject. 26 However, a brief glance at the chronology sketched above suf-
fices to show that image-sound relations are located at the center of a complex
fabric of technology, aesthetics, perception, worldview, and economics, whose
mixture of constants (physiological, physical, and some derived from the his-
tory of ideas) and variables (technical, cultural, and in the broadest sense ideo-
logical) cannot be depicted as permanent progress.

The history of the ideas of audiovisual synthesis is often far ahead of the history
of technology. Feasibility sometimes only catches up with utopias when their
most intense phase is already past. Absoluteness, which in Richard Wagner’s
day and again in the 1920s and 1960s was the basis for the demand for and
expectation of increasing synthesis in “the artwork of the future,” for the aboli-
tion of all boundaries between genres, and for universal audiovisualization of
aesthetics, today is obsolete. Yet the arts have undoubtedly undergone exten-
sive Verfransung (fraying). This metaphor of Theodor W. Adorno’s clearly shows
that although such fraying may blur the margins of the fields, it does not call the
core area into question. The countermovement to the fraying of the edges is the
conscious, radical return to one’s own genre, as, for example, Clement Green-
berg’s modernism demands. To Greenberg, culture that focuses on its own
medium is a bastion against capitalist kitsch that mixes all media and materials. 27

Rather than permanent progress, one can certainly describe the two-hundred-
year-old history of the color organ and related constructions by artist-inventors
right up to audio-video synthesizers as a history of permanent failure. The
search for an ideal, scientifically established, objective correspondence of col-
ors and sounds, which some of the color organs were intended to demonstrate,
proved to be unsustainable and not even capable of being universalized. It is
not possible to justify specific linkages of image and sound scientifically or aes-
thetically; ultimately, they are based on individual preferences. Although intui-
tive access to the quality or intensity of the linkage of sound and image
through direct experience of audiovisual culture is still possible, it is very diffi-
cult to abstract from this or make comparisons with other examples because
we are virtually unable to name this “third party” situated between hearing and
seeing or to subsume it under objectifiable criteria.

26 For an example of a typical genealogy of progress, see Peter Weibel, “Von der visuellen
Musik zum Musikvideo,” in Clip, Klapp, Bum. Von der visuellen Musik zum Musikvideo, eds.
Veruschka Bódy and Peter Weibel (Cologne: DuMont, 1987), 53–141.
27 See Clement Greenberg’s famous essay “Avant-garde and Kitsch,” Partisan Review 6 (1939):
34–39.
14

Comparable questions, which are just as irresolvable as those concerning per-

ception, apply to the area of the relevant apparatus. These are hybrids located
somewhere between a work of art, an instrument, and media technology. From
Father Castel to Thomas Wilfred, all of their inventors and constructors hoped
that they would proliferate on a massive scale. However, they do not possess
the ability to achieve intersubjective consensus regarding a work of art, nor the
instrumental universality of a musical instrument for very different types of
music. For this reason, these hybrid devices remained tied to the performances
of their creators for the most part; they often disappeared from the public eye
together with their constructors and are only documented in descriptions or
photographs. An additional complication is that neither the history of art,
music, or technology appears to feel responsible for such hybrid apparatus;
therefore, they are excluded from established institutions of conservation. This
history of apparatus continues to apply today to the abundance of audiovisual
software that has been developed, for which there is also no established con-
text of cultural evaluation or archiving. 28 Yet the success of digital technology
does relativize the aforementioned two-hundred-year history of failure. The
technical reproducibility and universal functionality of digital interfaces, such
as the Lemur by JazzMutant or Pioneer’s DVJ-X1, are heralds of instrumental
standards for the production of audiovisual artifacts. The same applies to the
user interfaces of software of this kind: the computer enables the universality
of applications, which are widely distributed as plug-ins and emulations and in
their turn influence the aesthetics of production.

In this sense, the history of artistic and technical sound-image linkages can be
regarded as an exemplary case for Adorno’s proposition that “progress in art
must not be denied; nor should it be proclaimed.” And Adorno sees the “dual
nature” of art—both social and autonomous—as the reason why it is “difficult to
talk of progress as both present and non-existent.”29 The dual nature of art can
also be confirmed for the dualism of art and technology, which is investigated
in this volume. Technological progress is undeniable, yet the history of ideas
about sound and image relations contains just as many examples of apparently
eternal, recurring motifs, which are as fascinating as they are ultimately not
entirely resolvable.

Therefore, the potential antithesis of a history of progress would be the ever-

recurring questions, motivations, and aims of the perennial work on image-
sound linkages. This could take the form of a conscious and intentional reprise,
a historical reference, and new interpretation, as already exist in the history of
art and history of music. In the history of image and sound relations, however,
there are numerous examples of artists and inventors hitting upon innovative
ideas and realizations without being aware that they are in fact part of a long
tradition. In the history of art, music, and technology, those concerned tend to
overestimate how innovative their work is. Particularly in the case of the color
organ, but also in the wider field of audiovisual arts and apparatus, the belief
that one is the first and only author of a specific idea is astoundingly pervasive.
Adrian Bernard Klein, who invented such apparatus himself and also authored
the first in-depth historical account of two centuries of color music, wrote in
1927: “It is an odd fact that almost everyone who develops a colour-organ is
under the misapprehension that he, or she, is the first mortal to attempt to do

28 On the hybridity of aims and contexts, see the chapter “Software Art” by Golan Levin in this
volume.
29 Theodor W. Adorno, Aesthetic Theory, trans. C. Lenhardt (London: Routledge, 1984), 298 and
300.
 15

so.”30 Up until the present day, audiovisual products and software continue to
be touted as absolute innovations and “revolutionary fusions of the senses.”

One of the main reasons for these permanent reinventions is the fact that a his-
tory of audiovisuology does not exist, because due to the fact that image-
sound couplings reside in a state of in-between, there has been no develop-
ment of a specific theory or aesthetics and no canons have been established. In
this respect, these reinventions only seem naive from a retrospective point of
view; in their particular artistic, aesthetic, and technical situation, they were
original and experimental, even when they were created outside any context of
historical awareness. The recognition of the deficit of historical situatedness of
one’s own praxis motivates some of these artist-inventors to undertake retro-
spective genealogical research and often makes them recognized historians of
their respective métiers. 31

Unlike in the history of art or music, explicitly historical citations do not refer to
a succession of epochs or styles, but often occur across all the historical peri-
ods and genres involved. For example, John Whitney, a pioneer of computer
animation with his algorithmic visual music, refers explicitly to Pythagoras’s
doctrine of harmony. 32 On the other hand, there were certainly historical phases
of intense concretion in the artistic and technical praxis of image-sound cou-
plings, where the zeitgeist coalesced with available media technology and
inspiration from the field of science. This was the case in the 1920s: the artistic
Absolute Film, psychological research on color and sound, and the technologi-
cal advances in radio and sound film were parallel developments, which at first
were independent but later came together in spheres of mutual interest. Here,
there was already the seed of an audiovisuology that spanned art, technology,
and science, for example in Georg Anschütz’s color-sound congresses and in
the Bauhaus environment. 33 In the 1960s, too, intermedia art, expanded cinema,
feedback video techniques, experimenting with drugs, and popular theories—
from Marshall McLuhan to Timothy Leary—were all combined in the spirit of
psychedelia. 34 In the 1990s, the club culture, analog sampling and scratching,
new digital audiovisual software and hardware, and the need for visual addi-
tions to electronic music all complemented each other to give birth to live visu-
als. While it is not possible to offer an exhaustive treatment of these phenom-
ena here, they nevertheless illustrate the permanent return of certain
fundamental motifs—some as intentional historical references, some as naive
reinventions as mentioned above.

30 Adrian Bernard Klein, Colour-Music. The Art of Light (London: Lockwood, 1926), 21. Kenneth
Peacock expressed much the same view: “Nearly every color-organ inventor in the nineteenth
and early twentieth centuries was under the delusion that he or she was the first to conceive
of color-music. Mary Hallock-Greenewalt is perhaps the extreme example. Her book is a self-
panegyric in which she claimed in the opening pages, ‘It is I who have conceived it [color-
music], originated it, exploited it, developed it, and patented it.’” See Kenneth Peacock,
“Instruments to Perform Color-Music: Two centuries of technological experimentation,” Leon-
ardo 21 (1988): 404.
31 The first standard work about color music is by Adrian Bernard Klein, who performed such
experiments himself and only later became aware of the considerable history of the subject;
see Adrian Bernard Klein, Colour-Music. The same applies today to VJing and audiovisual
software, whose development is initially documented by the developers themselves.
32 John Whitney, Digital Harmony. On the Complementarity of Music and Visual Art (Peterbor-
ough, NH: Byte Books, 1980), especially the chapter “Pythagoras Revisited,” 65ff.
33 The second Color and Sound congress held in Hamburg in 1930 was attended by psycholo-
gists, scientists, engineers, and artists such as Ludwig Hirschfeld-Mack, Zdeněk Pešánek, and
Baron Anatol Vietinghoff-Scheel; see Georg Anschütz, ed., Farbe-Ton-Forschungen, vol. 3
(Hamburg: Psychologisch-ästhetische Forschungsgesellschaft, 1931).
34 “ To this day, psychedelic art offers a suitable instrument for the analysis of synesthetic-artis-
tic experiences in a world influenced by new technologies.” Christoph Grunenberg, ed., Sum-
mer of Love: Psychedelische Kunst der 60er Jahre (Ostfildern: Hatje Cantz, 2005), 40.
16

Because such entirely diverse contexts and factors are always linked in audio-
visuology, it is very difficult to develop a classification or a chronology for
these fields. The cross-connections between the artistic genres and the scien-
tific disciplines create a kind of network. As mentioned above with reference to
Bruno Latour’s term from science studies, the in-between areas only become
visible through these socio-technical networks, which are mostly ignored from
the perspectives of the individual disciplines. This complex structure can possi-
bly described by the term “family resemblance”—a philosophical idea proposed
by Ludwig Wittgenstein in his critique of language. Family resemblance does
not develop in linear sequence like a chronology of progress, rather a unique
and original mixture is created through overlapping similarities and differences
that enables the determination of a typical similarity for which, however, there
are no fixed and unchangeable criteria.

For this reason, the structure of relations in audiovisuology can best be

described by the term “semantic network.” A semantic network both enables
and renders necessary a synchronic and a diachronic viewpoint, but for this
reason inevitably eludes classical forms of knowledge representation. It will be
achieved for this publication through the parallel forms of book and online plat-
form. The linear history of technological progress and the cyclical history of
ideas both have their raison d’être, although one of these alone cannot claim
validity without admitting the other perspective. Only then does the more pro-
found reason for the paradox of the perennial and topical nature of the subject
mentioned at the outset become clear. This paradox is merely the symptom of
the different models of a possible chronology that at once contradict and com-
plement each other. Thus, the audiovisuology presented here is not a new dis-
cipline but a meta-level on which the convergence and divergence of audiovi-
sual art forms, methods, and scholarly disciplines become visible.
Essays

Dieter Daniels, Sandra Naumann, Introduction Vol. 2, in: Dieter Daniels, Sandra Naumann (eds.),
Audiovisuology, A Reader, Vol. 1: Compendium, Vol. 2: Essays, Verlag Walther König, Köln 2015,
pp. 439 - 441
 439

Introduction
Dieter Daniels and Sandra Naumann

The guiding theme of the contributions in this volume is the relationship

between the auditory and the visual in art, media technology, science, and
­perception. The period covered spans from the eighteenth century to the pres-
ent, with a particular focus on the twentieth century. This leitmotif is developed
along four different thematic lines:
(a) the relationship between artistic genres and their respective aesthetic
theories with reference to painting, sculpture, music, literature, and film;
(b) the coupling of images and sounds in the audiovisual media and artistic
apparatus found in the realms of film, video, and immersive or interactive
installations, as well as in their historical antecedents in the color-light art of
the nineteenth and early twentieth centuries;
(c) the interplay between these techniques and human perception to the
point of potential boundary experiences of multimodal synthesis, such as
those that occur in the phenomena of cross-modality, embodiment, immer-
sion, and dissolution;
(d) the convergence and divergence of visual and auditory codes in v ­ arious
forms of cultural expression, which has been thematicized, for example, in
the artistic avant-garde of the 1920s, in the intermedia art of the 1960s and
digital media art since the 1980s, and in pop culture since the 1960s, as well
as in the reflection of pop culture in contemporary art since the 1990s.

The complexity and intricacy of the guiding theme is evidenced, for instance,
by the fact that none of the ten essays ever deal only with a single topic, but
rather always create links between different levels. For example, the questions
of the definition of the artistic genres, their rivalry in the paragone, and their
reciprocal influence—long-standing questions which Simon Shaw-Miller asks
with respect to the period from the late eighteenth to the early twentieth cen-
tury—are certainly a prelude to the dialectic relationship between visual and
auditory codes, as outlined by Diedrich Diederichsen in reference to pop culture.
The coupling of image and sound in media technology and the oft-forgotten
contributions of the avant-garde artist-engineers of the first half of the twentieth
century, which are recalled in Birgit Schneider’s archaeological inquiry into the
history of media technology, are likewise found at the interface between art
and natural science, as are the immersive artworks examined by Chris Salter,
given their connection with neurological research and studies on intermodal
perception. Like Schneider and Salter, Katja Kwastek also wonders whether it is
possible to draw boundaries between the technical apparatus, the performative
instrument, and the reception-seeking artwork—a question that the author
examines using the example of interactive art. The interplay between bodily
proprioception and the transformation of the audiovisual by means of media
440

technology—dealt with by Michel Chion in terms of his concept of ergo-audi-

tion—is an important component, in turn, of the interaction between artwork
and user or performer, explored by Kwastek, and of embodiment and immer-
sion, dealt with by Salter. Finally, Hans Beller’s analysis of classical film montage
deals with related aesthetic phenomena, which are also studied by Christian
Höller with reference to contemporary art and by Diederichsen with respect to
pop culture: strategies of adaptation or even ingratiation of the auditory and
visual levels encounter strategies of contradiction and unveiled incompatibility.

As already illustrated in the interdisciplinary compendium of this two-volume

publication on “audiovisuology,” this field of research comprises a terrain of
overlaps, intermediate zones, and interferences. It is thus not a new science,
but rather represents a new perspective on existing fields of knowledge; none-
theless, the new combinations created give rise to new knowledge contexts. 1
The aim, however, is not to establish a new discipline, but to open up and ren-
der more permeable the existing scientific disciplines. The field of audiovisuol-
ogy outlined here can therefore exist only in a context of permanent dialogue
between the fields of audiovisual theory mentioned above and in constant
comparison with contemporary artistic practice, which often develops more
rapidly than science.

Observation of the current state of research shows that in the realm of the
audiovisual, practice is substantially more advanced than theory. 2 Vice versa, it
also becomes evident, however, that the diversity and intensity of audiovisual
practice is an essential impetus for the increased interest in the history and
­theory of combinations of image and sound. Today, the coupling and trans­
formation of visual and auditory data by means of digital technology seems so
self-evident that it is often presented as an ahistorical innovation—almost as a
side-effect of the universal machine of the computer. Only against the back-
ground of the long history of struggles with audiovisual apparatus and media
(often developed by artists themselves from the nineteenth century onward)
does it become clear that the conceptual history of the audiovisual in turn
in­fluenced and shaped the history of its technology. Artistic (audio-)visions led
to technical innovations, whereas today they often are understood only as the
implementation of the potential contained in digital technology.

Nowadays, the integration of sound in the visual arts and the augmentation of
contemporary (especially electronic) music by visuals are taken for granted.
The pathos of a Gesamtkunstwerk, as championed by Richard Wagner at the
close of the nineteenth century; the notion of a universal aesthetic validity that
accompanied the endeavor to achieve a synthesis of the arts in the early twen-
tieth century and that found concrete form in the “absolute film” of the 1920s
and in the psychological and physiological color-sound research of the same
era; and the aspiration for multimedia totality and the suspension of all genre
boundaries, as formulated by intermedia art in the 1960s—all have today
yielded to the unquestioned self-evidence of the audiovisual. Seen historically,
the goal was always an overall design that encompassed not only the coupling

1 See Dieter Daniels and Sandra Naumann, eds., See This Sound: Audiovisuology Compendium.
An Interdisciplinary Survey of Audiovisual Culture (Cologne: Walther König, 2010).
2 This view was expressed by the jury of the Prix Ars Electronica Media.Art.Research Awards—
dedicated in 2009 to the topic of sound-image relations in audiovisual art—after having
viewed the entries: “The conclusion of the jury thus was that artistic practice in its multi-­
medial, multi-modal approaches to sound and image is further ahead than current theory in
this interdisciplinary sense.” Jury Statement Prix Ars Electronica Media.Art.Research Award
2009, by Dieter Daniels, Christoph Grunenberg, Cornelia Lund, Helga de la Motte-Haber, and
Christopher Salter, in “Sound-Image Relations in Audiovisual Art,” in CyberArts 2009,
eds. Christine Schöpf, Gerfried Stocker, and Hannes Leopoldseder (Ostfildern: Hatje Cantz,
2009), 241.
Introduction | Dieter Daniels and Sandra Naumann 441

of image and sound, but also a fusion of aesthetic theory and practice that
would lead to a utopian unity of sensory perception. There are no longer over-
all designs of this kind for today’s ubiquity of audiovisual art forms; rather, the
advanced lead in practice mentioned above evidences a deficit at the theoreti-
cal level. And yet the omnipresence of the audiovisual has led neither in artistic
practice nor in science to an ongoing discourse that spans all the genres. The
valuation contexts of visual art, music, theater, and film are as separate today
as they have ever been. Against this background, the compendium of audio-
visuology should be understood as an attempt to link the perspectives of the
individual scientific disciplines with one another.

The essays in this volume provide a critical review of the long search for possi-
ble syntheses between aesthetic theory formation and media technology prac-
tice. Whereas the individual art forms and technical procedures are presented
chronologically and systematically in the compendium, the essays draw cross-
connections between the four thematic levels mentioned above. The common
goal is to reawaken awareness of the unquestioning acceptance of our media-
based audiovisual environment as a matter of course, to once again render
audible and visible that we live in the midst of a permanent artistic coupling of
the auditory and the visual, and thus to allow both the possible synthesis and
the contradiction between images and sounds to become explicit. Here and
throughout the “See This Sound” project, the objective is to question the
­different artistic concepts, their models for value creation, and the relevant
­scientific disciplines and their concepts of truth. 3

3 See also the exhibition catalog See This Sound: Versprechungen von Bild und Ton /
See This Sound: Promises in Sound and Vision, Lentos Kunstmuseum Linz, eds. Cosima Rainer,
Stella Rollig, Dieter Daniels, and Manuela Ammer (Cologne: Walther König, 2009).
Prologue

Hybrids of Art, Science, Technology,

­Perception, Entertainment, and Commerce
at the Interface of Sound and Vision
Dieter Daniels

Dieter Daniels, Hybrids of Art, Science, Technology, Perception, Entertainment, and Commerce
at the Interface of Sound and Vision, in: Dieter Daniels, Sandra Naumann (eds.), Audiovisuology,
A Reader, Vol. 1: Compendium, Vol. 2: Essays, Verlag Walther König, Köln 2015, pp. 442 - 459
 443

For an interdisciplinary theory that seeks to link the perspectives of different

disciplines, the following questions as to methodology arise: 1
— According to which criteria and paradigms are contemporaneous phenom-
ena from different contexts comparable with one another?
— To what extent can certain phenomena be assigned to specific contexts
without sacrificing their multiple points of reference and their hybrid identity
to a far too simple categorization?
— Are the intentions of the authors (artists, inventors, developers) valid criteria
for where to situate their artefacts (works of art, devices, concepts, produc-
tions), or should their actual usage (as artwork, technological device, scien-
tific demonstration, entertainment) be the determining factor?

Like the 35 contributions in the Audiovisuology Compendium, the essays in this

second Audiovisuology Essays volume traverse the contexts of art, technology,
science, perception, entertainment, and marketing in multiple combinations and
relations. For example, Katja Kwastek examines the ambivalence of audiovisual
devices in their double role as an instrument and a work of art from the per-
spective of art history. From the point of view of media theory, Birgit ­Schneider
demonstrates the hybridity of audiovisual experiments: the same artefacts are
propagated by their authors partly with artistic, partly with technological, and
partly with scientific goals. Chris Salter links theories from physiology and neu-
rology with concepts of aestheticism to investigate artistic-sensual-technological
border areas, which he also explores in his own artistical practice. As Simon
Shaw-Miller’s differentiation of inter-, cross-, trans-, and multidisciplinarity within
the arts demonstrates, the hybridity of different art genres is also just as com-
plicated. 2 The manifold interactions between pop-cultural codes, their commer-
cial exploitation, and media-technological formatting are highlighted in Diedrich
Diederichsen’s essay, how they are reflected in visual art in Christian Höller’s.
The hybrid disposition of acoustic self-perception between the inside and out-
side world is the theme of Michel Chion’s contribution.

This may sound like a résumé of the “new obscurity” identified by Jürgen
Haber­mas, 3 or like the typically post-modern situation where the categorizations
of scientific positivism reach their limits just like the conceptual structure of
cultural theory that has developed since the Renaissance. However, in the fol-
lowing I shall argue that we are not dealing solely with the description of a cur-
rent situation. In the thematic field of audiovisuology, it has been apparent for
some time now that instead of the categorical demand for clear classification
(either—or), there exists an indeterminateness (neither—nor), which is intrinsic
to this phenomenon; not a deficiency, but instead an essential or genuine

1 In the Introduction to Audiovisuology Compendium, the paradoxes of an overall chronology

of the parallel thematic strands were presented. The present Prologue focuses on the possi-
bility or impossibility of assigning individual phenomena to a specific context and of sharply
distinguishing between categories. In this sense, it functions as hinge between the two
Audiovisuology volumes. Dieter Daniels, Sandra Naumann, eds. See This Sound: Audio­
visuology Compendium (Cologne: Walther König, 2010), 5–16.
2 See the essay by Simon Shaw-Miller in this volume and his remarks on “Hybridity and Purity
in Artforms,” in Simon Shaw-Miller, Visible Deeds of Music: Art and Music from Wagner to
Cage (New Haven: Yale University Press, 2002), 11–29.
3 Jürgen Habermas, Die neue Unübersichtlichkeit (Frankfurt am Main: Suhrkamp, 1985).
444

i­ndetermination. 4 In the following, this will be characterized by the concept of

hybridity, in full awareness that in this term manifold connotations from science
and cultural theory converge. 5 These different meanings of hybridity match the
spectrum of themes covered in this volume, because many phenomena and arte-
facts of audiovisuology defy univocal classification. In Bruno Latour’s science
studies, the hybrids (later he also refers to them as quasi-objects), “chimeras
between nature and culture,” take on a key role in his critique of the modern era’s
mania for categorization. 6 The intention here is to make this figure of thought
from science studies fruitful for cultural and media theory as a leitmotif, because
so far this thematic complex lacks an adequate method to deal with hybridity.
Therefore, we shall turn our attention to the monsters that cannot be sorted
into any species pigeonhole—the only thing is, we don’t deal with life forms, but
with devices.

My second proposition is that this genuine hybridity is based above all on the
development of audiovisual devices since the eighteenth century. Thus, the
analysis of image/sound relations can be classed as an exemplary case study
for the entire field of art/technology relationships, and as a fore­runner of issues
in contemporary media art. 7 The prehistory of a correlation between sound and
color reaches back to classical antiquity, and the practice of linking images and
sounds can actually be recognized as an anthropological constant. 8 For centuries
people sought correspondences between human perception and the physical
world order by constructing analogies (or conjuring up magical ones) between
the senses and the absolute. Embedded in a model of universal h ­ armony, which
in addition to color and sound also included the seasons, elements, planets,
metals, and points of the compass, this was all about the “big” questions, such
as the relationship between humans and nature in God’s plan, purportedly
reflected in a direct correspondence between the subjective intensity of the
senses and the objective character of nature. Access to these holistic ideal
truths was sought in very different ways, both using the mind and the senses.
Pythagoras’ “harmony of the spheres” or Musurgia universalis by Athanasius
Kircher are—although they have been disproved by modern physics—mathe-
matical models of a high order. On the other hand, mystical and ecstatic, para-
religious experience, which is supposed to lead to direct intuition, is often
­likened to the synthesis of hearing and seeing—from prehistoric rituals to
today’s rave culture. Theosophical and occult theories cite references that
range from Kircher to Kandinsky in their enthusiasm for synesthesia. 9 For his
light-music, Alexander Scriabin planned a multi-sensory temple of mysteries,

4 Cf. Irmela Schneider, “Hybridization follows . . . the logic of ‘as well as’ and not of ‘either—or.’
This kind of logic does not absolve one from the cognitive task of differentiating, without
which insight is impossible; however, it clearly demonstrates that thinking in alternatives and
opting for one or the other side is both a choice and a decision that is neither logically inevi-
table nor natural.” Irmela Schneider, “Von der Vielsprachigkeit zur Kunst der Hybridation,” in
ibid. and Christian W. Thomsen, eds., Hybridkultur. Medien, Netze, Künste (Cologne: Wienand,
1997), 14–66, here 45–46.
5 On the various usages of the concept of hybridity, see Schneider and Thomsen,
Hybridkultur, 1997; Gerfried Stocker and Christine Schöpf, eds., Hybrid: Living in Paradox.
Ars Electronica 2005 (Ostfildern: Hatje Cantz, 2005).
6 Bruno Latour, We Have Never Been Modern, trans. Catherine Porter
(Cambridge, MA: Harvard University Press, 1993).
7 Gaining access to history from the present viewpoint backwards, and an extension of media
art contexts are leitmotifs for the entire project of “See this Sound,” including the accompa-
nying exhibition; see Dieter Daniels and Stella Rollig, “Preface,” in See This Sound: Promises in
Sound and Vision, eds. Cosima Rainer, Stella Rollig, Dieter Daniels, and Manuela Ammer
(Cologne: Walther König, 2009), 10–13, here 12.
8 Cf. Dieter Daniels and Sandra Naumann, “Introduction,” in
Daniels and Naumann, A ­ udiovisuology Compendium, 6.
9 Cf. Andrea Gottdang, “Painting and Music,” in Daniels and Naumann,
Audiovisuology ­C ompendium, 246–257, here 251.
Prologue | Dieter Daniels 445

–T
 itle page of the essay “Clavecin pour les yeux,
avec l’art de Peindre les sons, & toutes sortes de
Pièces de Musique” by Louis-Bertrand Castel,
­M ercure de France (November 1725), 2552–2577.

which was never realized. Comparable knowledge about the world was prom-
ised by the combination of drug-induced experience and the psychedelic light
show environments of the 1960s. 10 Without such a metaphysical superstruc-
ture, the interactive immersion in computer games or in performances of live
visuals connects sensorimotor activity with audiovisual perception to produce
a synesthetic experience of presence.

Case Study Hybrid Artefacts: Aesthetic Evidence versus

Physical Experiment—Castel and Chladni
In the Age of Enlightenment a new chapter began for this long history of color/
sound correspondences. Almost all publications on this subject mention the
French Jesuit, mathematician, physician, and philosopher Louis-Bertrand
­C astel as a prominent forerunner of present-day developments. And indeed, a
few important innovations are found in Castel’s works: 11
— For the first time, a theory is formulated which refers exclusively to color/
sound analogies, and is no longer embedded in a holistic model for explain-
ing the world.
— For the first time, the attempt is made to bring the mind and the senses into
consonance. Castel’s model aspires to be mathematically, physically, and
aesthetically compelling.
— For the first time, a device is proposed that could serve as proof of the theory,
and as its practical application.

The role assigned to the device known as the clavecin oculaire (ocular harpsi-
chord) was key; if it worked, Castel’s hypotheses would be confirmed scientifi-
cally and rationally, as well as intuitively and sensually. To get straight to the
point: the ocular harpsichord, originally conceived by Castel as a thought exper-
iment, apparently never worked properly. Despite the extensive debates that
surrounded this device, no eye or ear witness accounts of a successful presen-
tation exist. Wisely, at first Castel was against constructing such an apparatus:
he said that he spoke only as a philosopher, not as a craftsman. 12 However, the
great public interest and the criticism of prominent contemporaries, such as
Diderot, Voltaire, and Rousseau, made him feel obliged to provide experimental
proof of his controversial hypotheses.

10 These mystical experiences of true insights, however, tend not to be sustainable; see Arthur
Koestler’s comment on drug-induced experiences to Timothy Leary: “I solved the secret of
the universe last night, but this morning I forgot what it was.” Timothy Leary, Flashbacks:
An Autobiography (Los Angeles: Tarcher, 1983), 61.
11 See Jörg Jewanski, “Louis-Bertrand Castel. The Clavecin oculaire (after 1723),” in
Daniels and Naumann, Audiovisuology Compendium, 83.
12 Jörg Jewanski, Ist C = Rot? Eine Kultur- und Wissenschaftsgeschichte zum Problem der
­wechselseitigen Beziehung zwischen Ton und Farbe. Von Aristoteles bis Goethe
(Sinzig: ­Studio, 1999), 283.
446

In the Age of Enlightenment, a hypothesis had to be tested and proved through

an experiment or demonstration, as Diderot demanded in his Encyclopédie with
regard to the effect that Castel ascribed to the ocular harpsichord: “Only direct
experience can decide this matter.”13 However, despite 30 years of frantic brico­
lage, the controversial theorist did not succeed in becoming a practitioner of
color music. In his fruitless efforts to make his natural-philosophical idea an
empirical and technological reality, Castel increasingly became the victim of his
own invention. 14 Moreover, it was not possible to prove the correctness of his
table of color/sound correspondences or indeed any of the experiments by
other researchers, which ultimately cancelled each other out because of their
diversity. 15 In this way, Castel also became the precursor of a leitmotif, which
runs through the entire history of ocular harpsichords and all later artistic and
technological experiments to visualize music: failure due to the lack of compat-
ibility between physical reality, theoretical insight, aesthetic vision, and techni-
cal feasibility.

Castel’s paradox lies in the fact that although he takes science as a starting point,
especially Isaac Newton’s Opticks, he does not formulate a clearly defined
rationale. The possible applications of his thought experiment seem to fascinate
him more than the proof of which colors correspond to which sounds. This is
already clear in the title of Castel’s first publication from 1725 and the sketches
of motifs for his invention which it included: practical, philanthropical uses
(deaf people could enjoy music through seeing it, blind people could perceive
colors through sound), educational use (schooling painters in the harmony and
dissonance of colors), its creative potential (a new instrument for the painting
layperson, who could effortlessly create thousands of pictures), and finally
purely aesthetic reasons (from capturing the fleetingness of music so it can be
analyzed at leisure with the eye, to decorating a space with a tapisserie harmo­
nique, which allows visual enjoyment of an entire piece of music). 16 Castel
prophesied that his ocular harpsichord would one day be as popular as tradi-
tional musical instruments, and in Paris alone he expected to sell 800,000 of
them. 17 Whether his apparatus is a scientific experiment, an instrument for a
new form of art, a medical prosthesis, a device for entertainment, or the
­prototype for a new branch of industry, is ultimately undecidable. 18

Castel’s approach is a crude mixture of physics, philosophy, physiology, aes-

thetics, and relics of theology. His ocular harpsichord was supposed to prove
physics through aesthetics; that is, the analogy of the materiality of light and
sound was to be explained through human perception of them. This indicates
that, ultimately, Castel stands in the tradition of the holistic world harmony
models, from Pythagoras to Kircher. From the point of view of science in the
age of empiricism, experiment, and enlightenment, this way of thinking in

13 Denis Diderot 1753, cited in Jewanski, Ist C = Rot?, 365.

14 Cf. Maarten Franssen: “A picture emerges of a man gradually worn out completely by his own
invention, although he kept believing in it to the last.” Maarten Franssen, “The Ocular Harpsi-
chord of Louis-Bertrand Castel: The Science and Aesthetics of an Eighteenth-century cause
célèbre,” in: Tractrix. Yearbook for the History of Science, Medicine, Technology and
­M athematics, 3, 1991, 15–77, here 28.
15 See the table by Jörg Jewanski in Daniels and Naumann, Audiovisuology Compendium, 345.
16 Louis-Bertand Castel, “Clavecin pour les yeux, avec l’art de Peindre les sons, & toutes sortes
de Pièces de Musique,” in Mercure de France, November 1725, 2552–2577.
17 Barbara Kienscherf, Das Auge hört mit: Die Idee der Farblichtmusik und ihre Problematik—
beispielhaft dargestellt an Werken von Alexander Skrjabin und Arnold Schönberg
(Frankfurt am Main: Peter Lang, 1996), 37.
18 For a viewpoint from science historians (“whether the ocular harpsichord was a scientific instru-
ment or not, depends on one’s point of view”) see Thomas L. Hankins and Robert J. ­Silverman,
Instruments and the Imagination (Princeton: Princeton University Press, 1999), 74.
Prologue | Dieter Daniels 447

– S ound patterns (1787) by Ernst Florens Friedrich Chladni.

Source: Ernst Florens Friedrich Chladni, Entdeckungen
über die Theorie des Klanges (Leipzig 1787), 115, plate X.

­ nalogies appears totally antiquated. 19 However, from the retrospective point of

a
view of cultural history, the futuristic aspects of Castel’s ideas become appar-
ent, and can today be read as a kind of science fiction.

The sound figures that were generated and described in 1782 by Ernst Florens
Friedrich Chladni are a counter-example to Castel’s ocular harpsichord. In his
experiments, sound was used to excite fine sand sprinkled on thin plates, which
visualized the vibrations as exquisite patterns and lines and permitted visual
analysis of the oscillations. The patterns were no longer based on speculative
analogies, but represented an objective correspondence between acoustic and
optical phenomena. From these premises Chladni, who was born one year
before Castel died, developed the physical basis of acoustics. His starting point
was clearly scientific: the oscillation of strings could already be calculated, so
Chladni wanted to explore the “true complexion of the sound of such bodies, in
which the elastic bending of whole surfaces in several dimensions at once
come into question.”20 The aesthetic fascination of the sound figures contrib-
uted significantly to the success of Chladni’s copiously illustrated books. He
also suggested using the figures to enrich the repertoire of patterns used in the
cloth and wallpaper manufacturing industries. 21 From 1789, Chladni also used
his discoveries to invent two new kinds of musical instruments, the Euphon and
the Clavicylinder, which especially enabled him to improve his precarious finan-
cial situation. He demonstrated the instruments himself in numerous concerts,
at which he also demonstrated the sound figures. 22

Both Chladni and Castel are part of a hybrid praxis. As authors and actors they
stand in their contemporary context between the realms of science, aesthetics,
invention of devices, and entertainment. Their linking of science and art, how-
ever, took place from reversed directions. Whereas Castel wanted to prove a
physically inexplicable analogy of color spectrum and musical scale via aes-
thetic evidence, Chladni analyzed in his experiments the physical structure of
sound waves in solid bodies, and from this early form of scientific visualization,
he derived scientifically valid experiments as well as artistic and entertaining
results. This casts Castel as a forerunner of the understanding and misunder-
standing of art as science, and Chladni, vice versa, as a forerunner of the
equally problematic science as art.

19 On Castel’s theological rhetorics of analogy, see: Hankins and Silverman, Instruments and
the Imagination, 80ff.
20 Ernst Florens Friedrich Chladni, Entdeckungen über die Theorie des Klanges, Leipzig 1787, 1.
21 See the work description of Chladni’s figures by Birgit Schneider in this volume.
22 “ The proceeds from his lecture tours and his works had to provide the means for his upkeep
and for his experiments.” Eugen Lommel in Allgemeine Deutsche Biographie, published by
the Historische Kommission bei der Bayerischen Akademie der Wissenschaften, vol. 4, 1876,
125; see also: Dieter Ullmann, “Life and work of E.F.F. Chladni,” in The European Physics
­J ournal, Special Topics, 145, 2007, 25–32,
online: http://www.springerlink.com/content/fx2jm482p0404q33/fulltext.pdf.
448

The interesting thing about Castel is not his misguided theory or his non-func-
tioning apparatus, but instead his attempt to link theory, sense perception, and
device. From this point onwards, the history of correspondences of the visual
and the auditive also becomes a history of technology. 23 Through technology,
the relation between optics and acoustics is no longer restricted to the color/
sound analogy; but the representation of its physical nature, its morphology so
to speak, achieves far wider dimensions. Dimensions with respect to instruments
and devices as well as scientific and aesthetic ones—this became apparent for
the first time with Chladni’s figures. In 1802, Chladni’s contemporary Thomas
Young succeeded in demonstrating the wave form of light. This laid the physi-
cal foundation for the development of audiovisual media technology in the
nineteenth century, and at the same time eliminated the basis for the centuries-
old quest to discover analogies in the phenomena themselves.

Up to this point in history, the suspected analogy between the natural phenom-
ena of sound and light was based on the purely subjective experience of a rela-
tion between hearing and seeing, as well as on holistic models of world harmony.
After Castel and Chladni, images and sounds were also coupled through
devices and experiments created by humans. On the one hand this coupling is
objective, because it is technical and physical, and on the other it is subjective,
because it is manipulable and controllable. This marks a new era in the linking
of image and sound, which extends from the development of optical and
acoustic media in the nineteenth century to contemporary universal possibili-
ties to modulate, generate, and transform the audiovisual by digital means.

Aesthetic, Epistemic, Pragmatic, and Entertaining Devices

The hybridity of science, art, entertainment, and commerce outlined here can
also be demonstrated specifically for the development of media technology.
This not only concerns the heterogeneous motivations for and contexts of cur-
rent inventions, but also the hybridization of the optical and acoustic processes,
whose development continues through combinations and permutations of their
functional principles.

The findings of basic research in physics and physiology since the beginning
of the nineteenth century (including work by Chladni, Young, and especially
­H ermann von Helmholtz’s extensive studies of physiology, optics, and acous-
tics) began to be utilized in the second half of the nineteenth century in specific
apparatuses and media. The epistemic device of the laboratory experiment,
which was originally constructed for research purposes, was translated into
media-technological applications suitable for everyday use that gave rise to an
audiovisual mass culture of pragmatic and entertaining devices. 24 Initially the
technological media separated the visual from the auditive. Silent films, the
gramophone, telephone, and early ideas for television all specialized in the
technological emulation of just one human sense faculty.

23 The music machines of the Baroque age can be regarded as precursors, for they comprised
both sound and moving figures, although they were also models for possible early industrial
production techniques: cf. Salomon de Caus, Von gewaltsamen Bewegungen: Beschreibung
etlicher, so wol nützlichen alß lustigen Machiner (Halle: Stekovics, 2003), reprint of the
­Frankfurt edition of 1615.
24 S ee in this context Hans-Jörg Rheinberger’s concept of “epistemic things,” which are based
on available technology, but in the context of experimental systems can also transcend it and
interrogate the basis of their own development; Hans-Jörg Rheinberger, Toward a History of
Epistemic Things: Synthesizing Proteins in the Test Tube (Palo Alto: Stanford University Press,
1997).
Prologue | Dieter Daniels 449

–L  issajous figures for various frequency

ratios, in different stages of their cycles.
From Koenig’s Acoustic Catalogue,
1865. Source: Case Western Reserve
University, Collection of Antique Phys-
ics Instruments.
– V ibration microscope for the observa-
tion of Lissajous figures (c. 1860) by
Hermann von Helm­h oltz, model from
Koenig’s Acoustic Catalogue, 1865.
Source: Case Western Reserve Univer-
sity, Collection of Antique Physics
Instruments.

However, both the history of the ideas and the operating principles of the opti-
cal and acoustic media were engaged in an ongoing dialogue. The invention of
the telephone by Alexander Graham Bell in 1876 supplied the inspiration for
Thomas Alva Edison’s Phonograph, and also led to plans for electronic trans-
mission of images because of the photo-electric sensitivity of selenium, which
was known since 1872. Basic concepts for the medium of television were formu-
lated around 1878 and envisaged the transmission of signals live via wires;
­however, this could not be realized due to the state of technology at the time.
The parallels between sound and image technologies were also evidenced by
Edison’s prototype for the Kinetoscope of 1888, which was nothing but a
Phono­graph fitted with chronophotographic images. 25 The formulation in the
patent, “to develop an instrument, which does for the eye what the Phono-
graph does for the ear,” can be taken quite literally. 26

The history of ideas for the transmission medium television and the storage
medium of film operate in the gap which had developed between image and
sound as a result of photography, telephony, and the Phonograph: if still images
and time-based sounds can be stored—and sounds can be transmitted elec-
tronically—why shouldn’t it be possible to transmit and store moving images,
too? Ever since, such conclusions by analogy between acoustic and optical
media have characterized the development of radio, television, and sound film
as well as the audio-video synthesizer. This is why it is wrong to reduce the par-
allel histories of each of the audiovisual media to separate lines of development
for images and sound. Rather, they should be understood as a complex interac-
tion, which already contains the potential for its multimedia synthesis.
The prehistory of this development of optical and acoustic media devices that
keep intersecting, is found in Hermann von Helmholtz’s research in optics and
acoustics. “This back and forth comparing the models of the two sensory
­systems” led him to the first comprehensive theory that relates the physical
characteristics of light and sound to the physiological faculties of sight and
hearing. 27 The laboratory instruments that Helmholtz developed played a key
role in this.

Helmholtz modified a telegraph constructed by his friend Werner Siemens and

around 1860 the vibration microscope was created. The instrument visualizes

25 In 1878, Edison was already thinking about connecting the playback of images and sound,
though it was not until he encountered Eadweard Muybridge and his Zoopraxiscope in 1888
that Edison’s assistant William Dickson modified a phonograph by adding 42,000 pictures
and the ocular of a microscope, and transformed it into an image machine; see Neil Baldwin,
Edison: Inventing the Century (New York: Hyperion, 1995), 211–212.
26 S ee Jan Philip Müller, “Synchronization as a Sound/Image Relationship,” in
Daniels and ­N aumann, Audiovisuology Compendium, 400–413.
27 Cf. Timothy Lenoir, “Farbensehen, Tonempfindung und der Telegraph: Helmholtz und die
Materialität der Kommunikation,” in Hans-Jörg Rheinberger and Michael Hagner, eds.,
Die Experimentalisierung des Lebens (Berlin: Akademie Verlag, 1993), 62.
450

– S ketch for the cylinder of the Peephole

Kinetoscope (c. 1888) by Thomas Alva
Edison. Source: The Thomas Edison
Papers, Rutgers, The State University of
New Jersey, Patent Series, Caveat Files:
Case 110: Motion P­ ictures (1888)
PT031AAA1; TAEM 113:238.
– P yrophone (1875) by Frédéric Kastner,
played by Wendelin Weissheimer.
Source: Harald Szeemann, ed., Der Hang
zum Gesamtkunstwerk (Aarau 1983), 199.

sound in the form of overlapping Lissajous figures. Experimental method and

the formation of theory proceed by constantly comparing auditory and visual
perception.

Through the vibration microscope various small phase differences of the par-
tials of more complex sounds become visible, although they do not influence
the tone color very much, as Helmholtz was able to demonstrate. This discov-
ery motivated him to work on developing Young’s color theory, according to
which color vision develops through comparable principles; namely, the recep-
tion of varying degrees of intensity within the spectral range of light. 28
This experiment led Helmholtz to a theory that takes into account what the
perception processes have in common, but also the differences between neu-
ronal receptors in the eye and the ear. 29 The theory demonstrates scientifically
why a direct analogy of color shades and sound colors is not possible. The eye
can perceive a mixture of colors only as a single color shade, whereas the ear
can differentiate between the spectral components of a sound. 30

Helmholtz’s vibration microscope not only linked visual and acoustic perception,
it was also a hybrid of science and media technology: an epistemic laboratory
instrument, which was based on the pragmatic telegraphy device by Siemens,
contained the functional principles of telephone and Phonograph already fifteen
years before the inventions by Bell and Edison. Helmholtz’s research was
­continued in 1873 by Emil Du Bois-Reymond who exchanged optic and auditory
nerves in a physiological thought experiment that also inspired many artist-­
inventors. 31

The hybridity of art, technology, science, and entertainment can be demon-

strated in many examples from the history of technology. One example is the
history of the origins of film, which culminates in the first public film shows in
Paris and Berlin in 1895, and earlier at the World Exhibition in Chicago in 1893. 32
These parallel inventions all have an individual prehistory: advances in the pho-
tographic industry (cinématographe by the brothers Auguste and Louis Lumière),

28 Thanks to Jan Thoben for his support in differentiating the argument on Helmholtz.
29 “ Through this the qualitative differences in the visual impressions are attributed to the
­various receiving nerves. Then there remains only the quantitative differences of stronger or
weaker excitation for the impressions of each optic nerve fiber. The same is accomplished by
the hypothesis for hearing, which was the result of our study of tone color.” Helmholtz cited
in Lenoir, “Farbensehen, Tonempfindung und der Telegraph,” 64.
30 H elmholtz summarized by Timothy Lenoir: “The eye does not know any kind of music,
because it only possesses three instead of the 1000 ‘resonator’ types of Corti’s membrane.”
Lenoir, “Farbensehen, Tonempfindung und der Telegraph,” 64. On the superceding of
­H elmholtz’s quantitative model by neurobiology, see the essay by Chris Salter in this volume.
31 On Emil Du Bois-Reymond, see the essay by Birgit Schneider in this volume.
32 O n the numerous parallel inventions see: http://www.victorian-cinema.net/machines.htm.
Prologue | Dieter Daniels 451

–A  lexander Wallace Rimington’s Color-Organ

­( 1895).Source: Adrian Bernard Klein, Colour
Music, The Art of Light (London 1926), plate 11.
– Page from patent for the Chromopiano
(1921/1926) by Arthur C. Vinageras.
Source: United States Patent US1577854,
http://www.freepatentsonline.com/.

new approaches to mass entertainment (the Bioscope of the showmen broth-

ers Max and Emil Skladanowsky in Berlin), and transfer of sound storage to the
moving image (Kinetoscope by Thomas Alva Edison).

Science provides the foundations for media technology and, in turn, technolog-
ical innovations trigger scientific debates. An example of this is the Phonograph,
which Edison invented in 1877, at first without assigning it a specific function.
Edison publicized its unlimited possibilities and made great efforts to
­demonstrate this with numerous examples. Amongst these uses were: singing
children to sleep, recording the last words of famous men, distributing audio
books in editions of millions, playing musical compositions backwards or
slower or faster, and, half-jokingly, recording men’s vows of love, so that the
women they cheated on could play this back again to the philanderers. 33 How-
ever, the Phonograph was initially an epistemic device, whose epistemological
implications made its inventor world-famous. Numerous reactions to it in the
USA, and even more in Europe, can be summarized in one question: when a
device de facto demonstrates what had previously been considered impossible
according to the world view of Aristotelian physics—namely, that the flow of
time could be stored, and could actually be played backwards—does this mean
that future progress in science, in philosophy, as well as in physiology and phys-
ics, can now only be achieved via technology? Analogies were made with the
functioning of human memory as hitherto the only storage medium for time. “
Is the brain a Phonograph?” was a question that was seriously discussed. 34
It took over 20 years before Edison was able to develop a commercial model of
the Phonograph from the patent.

The Pyrophone by physicist Frédéric Kastner was a comparable example of an

invention that was based primarily on aesthetic and philosophical motives, pre-
sented to the public for the first time in 1873. Like the Phonograph, it was
based on physical phenomena that had been known for some time. Colored
gas flames simultaneously generated the light and sound, utilizing the effect of
the so-called “singing flames,” which Bryan Higgins had discovered by chance
in 1777 and which were also researched by Chladni. The Pyrophone is a hybrid
of music and physics, of art and experiment. Henry Dunant, the philanthropic
visionary and founder of the Red Cross, who was financially supported by Kastner’s
mother, provided the natural-philosophical imagery for it, very much in the
­tradition of the holistic world models of previous centuries. For the parallel
generation of sound and light, Dunant employed the metaphors of harmonica

33 Cf. Edison’s article of 1878, cited in Baldwin, Edison, 403.

34 B aldwin, Edison, 439. On the phonograph as inspiration for the science fiction of an avatar in
Auguste de Villiers de L’Isle-Adam’s novel Tomorrow’s Eve, see Dieter Daniels, Kunst als
­S endung: Von der Telegrafie zum Internet (Munich: C. H. Beck, 2002), 68–75.
452

chimique and lumen philosophicum, 35 which are evocative of alchemy. Through

Dunant’s numerous lectures, the Pyrophone also aroused Richard Wagner’s
interest, who viewed it as a felicitous technical realization of his idea of the
Gesamtkunstwerk and wanted to use it in his operas. However, the bankruptcy
by Wagner’s patron, King Ludwig II. of Bavaria, prevented the realization of
these plans.

As the example of the Pyrophone shows, the history of media technology out-
lined above is accompanied by a parallel history of visual and auditory devices
by artist-inventors, most of which have been lost today. In the eighteenth cen-
tury, subsequent to Castel, several color organs were designed, although there
is no evidence that they were successfully realized. Then, from the mid-nine-
teenth century on, there was a long succession of devices for which their inven-
tors created new names—W. F. Philippy: Farbenklavier (1863); Bainbridge
Bishop: Color Organ (1876); A. Wallace Rimington: Mobile Color (1895); James
M. ­Loring: Musical Chromoscope (1900); Alexander Burnett Hector: Apparatus
for Producing Color Music (1912); Vladimir Baranoff-Rossiné: Piano Optopho-
nique (1916); Mary Hallock-Greenewalt: Sarabet (1918); Thomas Wilfred: Clavilux
(1919); Arthur C. Vinageras: Chromopiano (1922/1926); Ludwig Hirschfeld-
Mack: Farben Licht-Spiel (1922); Raoul Hausmann: Optophon (1922); Alexander
László: Sonchromatoscope (1925); Zdeněk Pešánek: Spectrofon (1926); Baron
Anatol Vietinghoff-Scheel: Chromatophon (around 1930).

Most of these devices were actually built and presented, but some were only
described or patented, though a few were even manufactured in small series.
The majority demonstrated color/sound analogies, some were also for playing
free-ranging image/sound compositions, and others just produced silent visual
music. Technologically the devices differed considerably, but were mostly a
combination of mechanical and electrical parts. Because of these technical dif-
ferences, the history of ideas that went into the instruments takes precedence
over their place in the history of technology. Paradoxically, this history of ideas
is not a continuous genealogy, but a story of multiple reinventions because the
authors rarely knew of each other’s existence. 36

Almost all of the artist-inventors expected a great future for their creations,
which were considered suitable for mass production and distribution, as had
been Castel’s intention. 37 These hybrids between instrument, work of art, and
media device, however, all shared a similar fate: they were dead ends. The com-
plicated apparatuses could only show their creators’ compositions, and not one
established itself as a standard instrument. These artefacts are the complete
opposite of universal machines: highly specialized, individualistic devices,
which therefore—metaphorically speaking—die together with their inventors
and are forgotten. None of the artist-inventors succeeded in getting his inven-
tion used, cared for, or developed by his successors, so that today only a few
working examples of such machines still exist. This proves the importance of
standardization and compatibility for the distribution and conservation of
audiovisual media, for which the 35-mm film, as the longest-living global media
format, is the best example.

35 [Henry] Dunant, “The Pyrophone,” in The Popular Science Monthly, August 1875, 444–453,
here 445. On Dunant and Kastner see Harald Szeemann, ed., Der Hang zum Gesamtkunstwerk
(Aarau: Sauerländer, 1983), 198.
36 S ee Daniels and Naumann, “Introduction,” 6.
37 Thomas Wilfred was one of the few who managed to sell a small series of sixteen models
of his Clavilux Junior (1930) for home use; see Yale University Library:
http://images.library.yale.edu/madid/oneItem.aspx?saveID=1776789&id=1776789.
Prologue | Dieter Daniels 453

The parallel development of audiovisual devices within the contexts of scien-

tific experiments, industrial media technology, innovative art, and broad-impact
mass entertainment illustrated here using individual cases, is the basis for the
suggestion to describe them as epistemic, pragmatic, aesthetic, and entertain-
ing devices. 38 What are the criteria for differentiation, though? Let us go back
to the comparisons mentioned above. From today’s perspective, the distinction
seems to be clear: Chladni’s figures are treated as a pioneering achievement in
acoustics by the history of science, whereas Castel’s ocular harpsichord is rele-
gated to the curiosities. Kastner’s Pyrophone has been largely forgotten,
whereas Edison’s Phonograph is mentioned in every history of technology. 39

Still, the motto of Chladni’s 1787 Discoveries Concerning the Theory of Sound is
“the art of painting with sounds,” a quotation from the poet Christoph Martin
Wieland. And to which category should the Phonoautograph be assigned, the
first machine for the time-based visual display of sound on a paper strip, pat-
ented in 1857 by Édouard-Léon Scott de Martinville, who had no idea that these
graphical traces of sounds were capable of being played back—something that
digital technology only made possible in 2008? The imaginative potential
unleashed by a system for two-way electrical transformation of picture and
sound signals is evidenced by the proposals of Maximilian Pleßner in 1892 for
hypothetical uses of future television technology that ranged from the artistic,
aesthetic, and analytical to the practical. 40 Let us expand the perspective to
include the present day, where the situation is even more opaque: the record-
ing principle of the Phonograph is taken by DJs in turntablism as a creative
technique for manipulating sound, not for reproducing it, which is why vinyl
records have survived into the digital age. And in the plasma tweeters of hi-fi
technology, the singing flames are used for the perfect reproduction instead of
the creation of music.

Issues of Method: Hybrid Identity, or Lost in

­Interdisciplinarity
But isn’t all this a misleading methodic mix-up? Is it legitimate to measure the
actual function of a device against the inventor’s or constructor’s intentions?
Shouldn’t the history of ideas be treated separately from the history of devices?
The technological artefacts themselves do not carry a telos within them; the
same functional principles can be used for very different purposes. In this
respect the motives of the inventors cannot represent criteria for the success
or failure of the artefact. Nevertheless, the history of ideas decisively influences
the actual implementation of technologies and their real applications.

We are now approaching an issue of methodology, for which Bruno Latour

coined the term “pragmatogony” to mean a mythical genealogy of the objects.
Pragmatogony describes an indissoluble, iterative interaction of social processes
and technological artefacts through which, according to Latour, the dualism of

38 O n the distinction between pragmatic and aesthetic devices see: Dieter Daniels,
“Sound & Vision in Avant-garde & Mainstream,” in Rudolf Frieling and Dieter Daniels, eds.,
Media Art Net 2: Key Topics (Vienna and New York: Springer, 2005), 59–87;
online: http://www.medienkunstnetz.de/themes/image-sound_relations/sound_vision/.
39 O n the hybridity of Pyrophony between science, art, and spectacle see Helmar Schramm,
“Pyrophonie: Anmerkungen zur Theatralität des Experimentierens,” in Helmar Schramm,
Ludger Schwarte, and Jan Lazardzig, eds., Spektakuläre Experimente: Praktiken der
­Evidenzproduktion im 17. Jahrhundert, Theatrum scientiarum, vol. 3 (Berlin, New York:
Gruyter, 2006), 398–413.
40 S ee Birgit Schneider’s description of Maximilian Pleßner’s brochure “Die Zukunft des
­e lektrischen Fernsehens” of 1892 in this volume.
454

– Still from the reconstructed color

version of Walter Ruttmann’s Licht­
spiel opus 1 (1921). © Eva Riehl,
courtesy Filmmuseum München.
– Page from the patent “Procedure
and device for production of cine-
matographic images” (1920) by Wal-
ter Ruttmann, with three movable
glass screens for the wet paint (c, d,
e), three illumination lamps (a), and
the camera (b). Source: Jeanpaul
Goergen, Walter Ruttmann. Eine
Dokumentation (Berlin 1989), 77.

technology and society is as impossible to uphold as the strict separation of

culture and nature, already sublated in Latour’s term “hybrids.” “But techniques
are not fetishes, they are unpredictable, not means but mediators, means and
ends at the same time; and that is why they bear upon the social fabric.”41 Prag-
matogony is intended to provide an alternative to the myth of progress; the
development of a field of knowledge that is demanded here through parallel
consideration of diachronic and synchronous depiction applies equally to the
thematic field of audiovisuology. Depending on the perspectives and case
studies selected, the history of acoustic and optical devices can be portrayed
either as permanent progress or as constant failure.

In its overall approach, audiovisuology aims to render the range of topics at

least halfway representable, despite the impossibility of constructing an exten-
sive chronology or methodology. Therefore, it is necessary that the disciplines
involved form a principled multi-perspectivity together, which has become
especially obvious in the chronological descriptions of individual phenomena in
the Audiovisuology Compendium. The thematic cross-sections in this second
volume present the plurality of methods that can be applied. As explained in
the Introductions to the two volumes, there is no chronology or method that
can claim any form of general validity. Further, the genuine hybridity of the
object of research, mentioned at the beginning of this Prologue, cannot be
entirely resolved through scholarship. 42

The indissolubility of this hybridity is also the main reason for what one could
describe as being lost in interdisciplinarity. On one side, this concerns the
­cultural and scientific evaluation of individual phenomena (artworks, devices,
theories), which, depending on their location within an art genre (music, paint-
ing, sculpture, film, and so on), in media technology, or in science, are subject
to entirely different evaluation criteria. It also concerns the absence of an
audiovisual historiography and, therefore, the handing down of knowledge and
the formation of cultural and intellectual traditions. This is the reason why many
color organ inventors believed that they were the first to have the idea of link-
ing hearing and seeing in an apparatus. 43

41 Bruno Latour, Pandora’s Hope: Essays on the Reality of Science Studies

(Cambridge, MA: Harvard University Press, 1999), 197.
42 This is the reason, why genuine hybridity also resists the holistic world harmony models and
the universalism of a Gesamtkunstwerk; see: Hans Ulrich Reck, “Entgrenzung und Vermischung:
Hybridkultur als Kunst der Philosophie,” in Schneider and Thomsen, Hybridkultur, 91–117, here
91. “Hybrid culture means the linking of contexts and areas that were originally separate into
something new, which precisely does not have the effect of dissolving the elements in a
synesthetically closed Gesamtkunstwerk, but in its aspects of divisions reveals an arrange-
ment that is still recognizable, that represents the dispositif of a montage, and whose effect
cannot be broken down into these parts.”
43 Cf. Daniels and Naumann, “Introduction,” 6.
Prologue | Dieter Daniels 455

–C  ircuitry for an electric apparatus

generating sound frequencies from
colored light (1931) by Walter
Brinkmann. Source: Walter
­B rinkmann, “Spektralfarben und
Tonqualitäten,” in Georg Anschütz,
ed., Farbe-Ton-Forschungen, Vol. 3
(Hamburg 1931), 358.
– S ketch of the 1919 version of the
Optophone by Raoul Hausmann,
made in the 1930s. Source:
­Leonardo 34, no 3 (2001), 218.

Case Study: Socio-Technological Networks. Absolute Film,

Radiophonic Art, Electrical Engineering, Anthropology, and
Synesthesia Research in the 1920s in Germany
With the onset of the twentieth century, the hybrid phenomena, which we have
investigated so far using examples from the eighteenth and nineteenth centu-
ries, became integrated in an increasingly dense set of relationships. We are
now no longer confronted with a solitary protagonist among the scientists,
inventors, and artists, instead, the motif of linking and transforming the visual
and the auditive runs through a wide spectrum of heterogeneous contexts. This
change created socio-technological networks, which according to Bruno
Latour evade being pinned down by the separated academic disciplines; how-
ever, the effects of these networks are real and considerable. An example is the
situation in 1920s Germany: absolute film can be viewed as the end and dissolu-
tion of the history of color organs, in that for many artists the medium of film
superceded creating their very own devices. The cinematographic apparatus
was modified, for example, by Walter Ruttmann and Oskar Fischinger to meet
the needs of their abstract films, so the history of the artist-inventors continued
into the medium of film. In parallel, radiophonic art was being developed for
the new medium of radio. The interaction between the aesthetics of silent mov-
ies and blind radio is demonstrated paradigmatically in Kurt Weill’s theory of a
non-narrative, acoustically abstract “absolute radio art,” which he formulated in
1925 with direct reference to absolute film. His intention was to “think through
to the end the often used and far too often misused comparison of film and
radio-broadcasting once and for all.”44 The most famous example of radio-
phonic art, however, was by Walter Ruttmann, pioneer of the absolute film: in
1930, Ruttmann produced the audiomontage Weekend, commissioned by Ger-
man Radio Broadcasting, using the Tri-Ergon process developed in Germany in
the 1920s, which inscribes sound as a light track on the edge of film stock. This
process was the first to store sound and image together on the same medium.
The technology was intended for synchronization, but could also be used for
artistic experiments in which visuals were transformed into acoustics. For the
first time, it enabled a free synthesis of sounds, as well as a direct analogy
between optical and acoustic perception. It was explored by Oskar Fischinger
from the perspective of film art, from the vantage point of an engineer by
Rudolf Pfenninger through experiments in synthetic sound. 45

The complex web of partially parallel, partially related developments of a

socio-technological network outlined here, is also embodied in exemplary

44 Kurt Weill “Möglichkeiten absoluter Radiokunst,” in idem., Musik und Theater: Gesammelte
Schriften, eds. Stephen Hinton and Jürgen Schebera (Berlin: Henschel, 1990), 192.
45 Cf. Thomas Y. Levin, “‘Tones from out of Nowhere’: Rudolph Pfenninger and the Archaeology
of Synthetic Sound,” in Grey Room 12 (2003), 32–79.
456

–G
 eneration of sound patterns of classical music on the screen of a
­N ipkow television system (1930) by Fritz Wilhelm Winckel.
Source: Fritz Wilhelm Winckel, “Vergleichende Analyse der Ton-Bild-
Modulation,” in Fernsehen, no. 4 (Berlin 1930), 171–175, here 173.

i­ndividual hybrid objects. A particularly incisive case is the Optophone. This

device was developed in the 1910s to enable blind people to “see”—a photo-
electric cell converted different light intensities, printed letters for example,
into a series of sounds. In the 1920s the Dada artist Raoul Hausmann developed
“Optophonetics” as a new form of art. He designed an appropriate device that,
when played as a live-instrument, would simultaneously produce images and
sounds, extending the artist’s sound poetry into a further medium. Hausmann’s
highly-detailed technical concepts were based on extensive research in physi-
ology and electrical engineering and led to an initially unsuccessful application
for a patent. 46 Through his collaboration with the radio and electronics engi-
neer Daniel Broido, Hausmann’s synesthesia device transmuted into an optical-­
mechanical calculating machine, which could be used to calculate the price of a
train ticket, for example, as stated in the new patent specification that was
granted in England in 1936. 47 It is highly doubtful whether Hausmann ever actu-
ally built an Optophone. Therefore, with regard to the multiplicity of its possi-
ble contexts and uses, the Optophone is a worthy successor to Castel’s ocular
harpsichord: both devices probably never existed as functioning machines, but
nevertheless sparked extensive debate.

Thus in 1927 the Bauhaus artist László Moholy-Nagy and the engineer Walter
Brinkmann, whom he quoted at length, both refer to the Optophone. 48
Expressly dissociating themselves from the color/sound analogies put forward
since Castel, they proposed to develop “scientifically based Optophonetics” by
using electrical waves as the carriers of both light and sound. At the experi-
mental radio workshop of the Musikhochschule in Berlin, Brinkmann developed
a device for “converting colored light effects . . . into audio-frequency electrical
oscillations with the object of producing musical sounds.”49 The goal is to find
“a basis for creating synesthetic art,” and thus to achieve “an approximate
agreement between empirically derived findings and artistic interests as the
precondition for a real color/sound art that will matter to a great number of
people.”50 In 1930 Fritz Wilhelm Winckel, a student of telecommunications and
acoustics, engaged with related issues in the private laboratory of Dénes von
Mihály. Winckel’s research was no longer based on the photoelectric cell but on
the new technology of television. The results of his experiments feeding electri-
cal acoustic signals into the new image medium were similar to Chladni’s sound
figures. However, his fascination with these figures motivated Winckel to pro-

46 For a more detailed depiction of the Optophone and “the multi-layered, often contradictory
concepts in art, technology, and science,” see the essay by Birgit Schneider in this volume.
47 Cf. Cornelius Borck, “Blindness, Seeing and Envisioning Prosthesis: The Optophone between
Science, Technology and Art,” in Dieter Daniels and Barbara U. Schmidt, Artists as Inventors—
Inventors as Artists (Ostfildern: Hatje Cantz, 2008), 109–129.
48 L ászló Moholy-Nagy, Malerei, Fotografie, Film, Bauhausbücher vol. 8 (Mainz and Berlin: Mann,
1967), reprint of the 1927 edition, 20–21.
49 Walter Brinkmann, “Spektralfarben und Tonqualitäten,” in Georg Anschütz, ed., Farbe-Ton-
Forschungen, vol. 3, (Hamburg: Psychologisch-ästhetische Forschungsgesellschaft, 1931),
355–365, here 355.
50 Ibid., 361.
Prologue | Dieter Daniels 457

pose a hypothesis about the objectification of beauty through the “synthesis of

art by electrical means,”51 which stated that “the individual character of an art-
work is contained in the modulation curve.”52 Such theories of a new aesthetic
by technicians may sound bizarre, but they were not without their counterparts
in the field of contemporary humanities.

From the viewpoint of his “philosophical anthropology,” Helmuth Plessner

developed a theory of “the unity of the senses” to relate in a more associative
way the “conceptions, ways of seeing and feeling of one art genre to those of a
different art genre.”53 The counterpart to Plessner’s subtle reflections on the
philosophical positioning of the human race and the special place it occupies
among living creatures, was the research conducted by Georg Anschütz. Based
on experimental psychology, Anschütz investigated color/sound combinations,
organized four congresses between 1927 and 1936, and published three sub-
stantial volumes that took in areas far beyond the core subject of psychology. 54
The Second Color/Sound Congress in 1930 in Hamburg was attended by psy-
chologists, scientists, and cultural studies scholars as well as artists—Ludwig
Hirschfeld-Mack, Zdeněk Pešánek, and Baron Anatol Vietinghoff-Scheel. A “sci-
ence and art exhibition” with a program featuring works by synesthesists, film
screenings of works by Oskar Fischinger, and a planned demonstration of the
apparatus constructed by Walter Brinkmann, attracted around 2,000 visitors.
The aim of this considerable undertaking, however, remained curiously vague.
As Georg Anschütz remarked in his introduction, color/sound research incor-
porates “the peripheral and the central, the sensory and the intellectual.” To
accomplish its purpose it needs to bring about a “vision” (which is not specified)
from a “mystical and dark sphere and recognize that it is something intrinsic to
all human beings, it permeates and rules our entire thinking, endeavors, and
work.”55 Anschütz’s call for “a new synthesis of mind” and “a new type of human”
are reminiscent of the holistic quest for world harmonies; however, as they
were supposed to arise from “the primordial and healthy mental force of our
people” his career under National Socialism is hardly surprising. 56

In the examples discussed above, a tendency can be found which objectifies

aesthetics scientifically and operationalizes beauty technically through the syn-
thesis of image and sound in electrical oscillations. In the 1960s there is a con-
tinuation of this in cybernetics and computer-generated creation or art analysis,
for example, in the work of Max Bense. Theodor W. Adorno and Hanns Eisler
had already criticized this tendency with reference to absolute film and color/
sound music as “speculations that seek to develop laws from the abstract
nature of the media as such, for instance from the relation between optical and
phonetical data . . . If artistic beauty is derived exclusively from the material of
the given art, it is degraded to the level of nature, but does not thereby acquire

51 Fritz Wilhelm Winckel, Technik und Aufgaben des Fernsehens (Berlin: Rothgiesser & Diesing,
1930), 59; on Winckel see the detailed essay by Birgit Schneider in this volume.
52 Fritz Wilhelm Winckel, “Vergleichende Analyse der Ton- und Bildmodulation,” in
Fernsehen 1, 1930, 171–175.
53 H elmuth Plessner, Die Einheit der Sinne: Grundlinien einer Ästhesiologie des Geistes
(Bonn: Cohen, 1923), 106.
54 G eorg Anschütz, ed., Farbe-Ton-Forschungen, Vol. 1, (Leipzig: Akademische Verlagsgesell-
schaft, 1927); Anschütz, Farbe-Ton-Forschungen, Vol. 3; Georg Anschütz, ed., Farbe-Ton-
Forschungen, Vol. 2, (Hamburg: Psychologisch-ästhetische Forschungsgesellschaft, 1936); cf.
Jörg Jewanski, “Kunst und Synästhesie während der Farbe-Ton-Kongresse in Hamburg 1927–
1936,” Jahrbuch der Deutschen Gesellschaft für Musikpsychologie 18 (2006): 191–206.
55 A nschütz, Farbe-Ton-Forschungen, V, VI.
56 G eorg Anschütz, “Die neue Synthese des Geistes,” in idem Farbe-Ton-Forschungen, 315–316.
From 1936 Anschütz was director of the office for the promotion of young teachers in the
Nazi association of lecturers and from 1939 leader of the Nazi district association of lecturers
(Gaudozentenbund).
458

natural beauty.”57 The question of where to draw the boundaries between

nature and culture, which had accompanied this thematic complex ever since
Castel and Chladni, finds its continuation in this context.

Perspective: Hybrid Artefacts in Socio-Technical Networks

In the Weimar Republic a multilayered network of media, art genres, and aca-
demic disciplines grew up around image/sound relations. There was extensive
interaction between artistic and technical media: painting, music, and sound
poetry met film, sound film, radio, and television. Furthermore, there were
inventions, like Hausmann’s Optophone and Brinkmann’s apparatus. An inter-
disciplinary diversity of aesthetic and technical competence was involved here:
painters became filmmakers and inventors of technical devices (Ruttmann),
musicians and filmmakers became pioneers of the radio play (Weill, Ruttmann),
artists worked with electrical engineers (Moholy-Nagy and Brinkmann, Haus-
mann and Broido), psychologists analyzed films (Anschütz and Fischinger), and
engineers proposed art theories (Winckel). The scientific contexts included
philosophy, anthropology, art and music theory, experimental psychology,
physiology, acoustics, and electrical engineering. This description covers just
one country (Germany) during one decade; it documents how concentrated
and networked the situation was, and clearly it is not possible to break the situ-
ation down into the categories art, technology, science, and media industry
without forfeiting its inherent dynamics and its significance. Yet even for this
relatively well-documented chapter of German cultural and media history an
adequate interdisciplinary account does not exist.

Bruno Latour coined the term socio-technical networks for such complex over-
lappings of scientific and scholarly disciplines, whereby he especially refers to
the separation of culture and nature. According to Latour, it is within these net-
works that so-called hybrids emerge to defy modern scientific categorization,
because the networks are not discernible from the given perspectives of the
separate disciplines. 58 Especially in the area of audiovisuology we are con-
fronted by such networks since the late nineteenth and early twentieth centu-
ries. We no longer meet with singular artefacts as curiosities, like the color
organs, or laboratory experiments like Chladni’s sound figures, or the Phono­
autograph, but instead we encounter a multiplicity of phenomena and artefacts
that cross-reference each other and that originate from completely heteroge-
neous social, cultural, and scientific contexts.

Following the historical development outlined above, it becomes clear that the
roots of hybridity reach back to the eighteenth century, and that audiovisual
devices play a key role, because these artefacts function as hinges and estab-
lish links between different contexts. However, the problem is not historical;
rather, it is a situation that remains unchanged today: its complexity is increas-
ing over time with the proliferating technical possibilities, especially where
electronics serve as the link between image and sound. 59 This has resulted in
contemporary practice being more advanced than theory, as mentioned in a
review of existing literature on the topic in the Preface. Building on a historical

57 Theodor W. Adorno and Hanns Eisler, Composing for the Films (London: Continuum, [1947]
2005), 64–65.
58 According to Latour these socio-technical networks are “simultaneously real, like nature,
­n arrated, like discourse, and collective, like society” and therefore represent an unresolvable
contradiction for modern scientific thinking; Bruno Latour, We Have Never Been Modern, 6.
59 O n the role played by electronics in the 1950s and 1960s see: Dieter Daniels, “From Visual
Music to Intermedia Art,” in: Rainer, Rollig, Daniels, and Ammer, See This Sound: Promises in
Sound and Vision, 240–253.
Prologue | Dieter Daniels 459

basis, the goal of Audiovisuology is to somewhat reduce this gap between

­theory and contemporary practice.
Just how topical the cited historical characteristics of hybridity are for current
practice is illustrated by this quotation from Golan Levin’s contribution on soft-
ware art from the first Audiovisuology volume:

Such works are produced for diverse social contexts and can serve a variety
of objectives. In the field at large, and in the examples discussed in this arti-
cle, software artworks serve some of the same aims as do cinema, perfor-
mances, installations, interior design, games, toys, instruments, screensavers,
diagnostic tools, research demonstrations, and even aids for psychedelic
hallucination—though many projects blur these boundaries to such an
extent that categorization may not be very productive. Likewise, audio­
visual software artworks continue to emerge from plural and only occasion-
ally intersecting communities of research scientists, new media artists, soft-
ware developers, musicians, and i­solated individuals working outside the
institutions of the laboratory, school, museum, or corporation. 60

Again, let us call to mind the multiplicity of applications and contexts that C
­ astel
envisaged for his ocular harpsichord. Two and a half centuries later, the genuine
hybridity of devices and artefacts at the interface between hearing and seeing
reaches far wider circles and contexts. Yet their acceptance is by no means a
foregone conclusion. A deliberate rejection of self-classification is still subject
to strong pressure in art, science, and media technology. Latour describes the
following paradox: “The modern Constitution allows the expanded proliferation
of the hybrids whose existence, whose very possibility, it denies.”61

The many hybrid devices, which emerge at the interface between the acoustic
and the visual, are exemplary for this conflict in the modern era. On the one
side they are part of the positivist history of progress and the ongoing process
of differentiation in art, science, and technology in the narrative of the modern
era. The propositions discussed above that aim to operationalize the arts as
electrical oscillations, are symptoms of such a belief in technocratic feasibility.
On the other side, the contexts of the creation of these artefacts frequently
reveal a longing to recover a pre-modern wholeness. This also drives the suc-
cess of image/sound synthesis in pop culture and the great interest in scientific
research on synesthesia. The search for wholeness can turn back to holistic
models of world harmony and lead to a theological, occult, spiritual, or drug-­
induced escape attempt from modernity. 62 As the essay by Chris Salter in this
volume illustrates, however, recent theories of neuroplasticity posit a dynamic,
sensorimotor concept of the interlacing of body, self, and environment, which
has been demonstrated for the cross-modal circuitry of vision and hearing. 63

Thus the thematic field’s genuine hybridity also transcends the opposition of
modern and anti-modern. The goal of Audiovisuology is not to establish a new
scientific discipline, but to outline a model for dealing with this hybridity, to
sustain it with open eyes and ears, and to withstand the temptation to con-
struct fallacious syntheses.

60 G olan Levin, “Audiovisual Software Art,” in: Daniels and Naumann, See This Sound: Audio­
visuology Compendium, 270–277, here 270.
61 Latour, We Have Never Been Modern, 34.
62 In the section “A Perverse Taste for the Margins,” Latour describes how the moderns and
­a ntimoderns “frighten each other by agreeing on the essential point: we are absolutely differ-
ent from the others, and we have broken radically with our own past.” Latour, We Have Never
Been Modern, 124.
63 This is not only found in persons who have lost a sense faculty through injury, but can also be
demonstrated in non-impaired test persons; see the essay by Chris Salter in this volume.