ContemporaryMusic Review 9 2000OPA (OverseasPublishersAssociation)N.V.

2000, Vol.19, Part 2, p. 61-80 Published by licenseunder

Photocopyingpermitted by licenseonly the Harwood AcademicPublishersimprint,
part of Gordon and BreachPublishing,
a member of the Taylor& FrancisGroup.

Technology and Creation-

The Creative Evolution

Eric Daubresse and G6rard Assayag

Translated by Joshua Fineberg

KEY WORDS: computer music; IRCAM; Fineberg; Empreintes; Hurel; Ler de choses;
Malherbe; Locus; Murail; L'Esprit des dunes.

History of the earliest avatars

F r o m its start, in t h e 1950's, c o m p u t e r m u s i c 1 h a s g e n e r a t e d e n o r m o u s

h o p e s c o n c e r n i n g its p o t e n t i a l to r e s o l v e m o s t of t h e p r o b l e m s l i n k e d to
p r o d u c i n g a n d e x p l o r i n g u n p r e c e d e n t e d l y n e w s o u n d s . 2 T h e efforts o f
t h i s field w e r e m a i n l y d i r e c t e d at r e s o l v i n g t e c h n i c a l p r o b l e m s ; h o w e v e r ,
w h i l e it b e c a m e t e c h n i c a l l y p o s s i b l e to s y n t h e s i z e a n y i m a g i n a b l e s o u n d ,
a more and more uncrossable barrier simultaneously appeared on this
n e w h o r i z o n . T h e s e s a m e s o u n d s t u r n e d o u t to b e m u c h less i n t e r e s t i n g
o n c e t h e y w e r e p l a c e d w i t h i n a m u s i c a l project; t h e y w e r e t o o a w k w a r d
to c o n c e i v e a n d t o o d i f f i c u l t to o r g a n i z e a n d v a r y w h i l e still p r e s e r v i n g
t i m b r a l c o n t i n u i t y a n d a s e m b l a n c e of i n t e r p r e t a t i o n .

1. translator's note: I have translated the French term 'informatique musicale' as computer
music throughout this article; however the real meaning is somewhat broader then the
English term. It implies all aspects of music that involve computer technology from com-
puter assisted composition and computer notation to sound synthesis and real-time elec-
tronic performance techniques.
2. translator's note: The French word used is 'inouis' which implies not just the mild sense
of new, but the unprecedented newness of something which has never before existed, or
perhaps even been dreamed of.
61
62 Eric Daubresse and G~rardAssayag

Composers interested by these newly offered possibilities found them-

selves powerless once they sat down in front of a computer. They spent
long hours learning programming languages, followed by whole nights
waiting for the results of a synthesis calculation that would often have to
be simply thrown away the next morning. Beyond these material
difficulties, they felt as if this type of work was a thousand miles from
any artistic endeavor - - cut off from musical tradition and their personal
knowledge and tools. The feeling that it should finally have become pos-
sible to control all the phases of creation (being simultaneously the com-
poser, the instrument builder and the interpreter) only aggravated their
frustration.

Spectral composers and the computer

In parallel with these technical developments, the musical trends of the

XXth century continued their diverse evolutions and one of the most
characteristic manifestations is the spectral movement. This movement
provides us with the occasion to answer certain questions that have
come out of research and will, a few years later, propose a musical and
aesthetic counterpart to the scientific work. (One might suppose that the
spectral domain only corresponds to perceptual phenomena. This is not
so. The spectral representation exists outside of any perceptual appara-
tus. It is fundamentally defined by the Fourier transform.) Effectively,
studying with a predilection for timbre and some of its characteristics,
the spectral composers immediately integrated discoveries concerning
sonic phenomena - - some of which they knew in only heuristic form.
They formalized these discoveries and then made use of them as soon as
they started engendering musical material.
Their compositional techniques were already sufficiently rich and
sophisticated to make them at ease in front of both analyses and synthe-
sizers; they went from an acoustical and musical multi-representation to
the programming of processes for the generation of symbolic or sonic
material. Manipulating timbre - - but also traditional instruments - - with
ease, freed from repetitive calculation, they certainly gave synthesis some
of its first proofs of musical respectability.

The unavoidable computer

The field of computer music has considerably developed since this time,
especially in terms of power, speed and conviviality. At the same time, it
 Technology and Creation 63

has become considerably less expensive. Personal computers have

become affordable for individuals, while simultaneously acquiring com-
puting p o w e r that is at least equal to that of the professional work-
stations used until recently. Programs dedicated to computer assisted
composition and analysis-synthesis (in particular), previously very slow
and hard to use, have become more ergonomic, open, modular and have
been enriched by the experience of composers, w h o have had more and
more ease of access to these applications.
In the technological domain, the research teams at IRCAM created a
workstation which was principally dedicated to real-time musical appli-
cations. They specially developed a card for signal treatment, which was
implemented for the NeXT computer, and which represented an enor-
mous gain in computing power. (This station replaced the 4X station - -
IRCAM's first real-time processor, which had become obsolete. The new
station did this at almost a tenth of the cost.) The development of soft-
ware dedicated to this platform followed - - especially for real-time. One
of these programs, the environment MAX-FTS was used in the creation
of all the works discussed in the second part of this article, either to
create the sounds or for performance in concert.
However, changes also took place in other fields of research; particu-
larly in the cognitive aspects. These have turned out to be decisive for
understanding musical sound: from its fabrication to its perception.
For composers, the computer has become, bit by bit, an irreplaceable
working tool; as important as the pencil and eraser. It fills several func-
tions, from printing out scores to playing back synthesized sounds in
real-time, and passing through the development and use of composi-
tional algorithms, simulating orchestrations or even controlling synthe-
sizers (virtual or external). It is important to remember h o w much
computers have profoundly changed not only the daily work habits of
musicians and scientists, but also, indirectly, in the ways that they con-
ceive of sonic phenomena, speculate on their possible manipulations, for-
malize and even express and communicate their ideas. These different
elements have allowed a giant qualitative leap in analysis, synthesis and
computer assisted composition.

The relations between composer and scientist

Certain skeptics see the notion of musical technique as disappearing,

making way for a technology which crushes all creativity. We, however,
will show, with some examples, h o w different composers, each through
his own preoccupations, has caused advances in research or has succeeded
64 Eric Daubresse and G~rardAssayag

in constructing a musical project w h i c h could not have been realized

without the back and forth between creative evolution and scientific or
technical research. One can observe that when there have been conver-
gences between the sensibilities or preoccupations of composers and sci-
entists, it is rare not to achieve positive results both in musical creations
and scientific research.
The position of 'compositeur en recherche '3 was created a number of
years ago at IRCAM. This term signifies composers who are integrated,
for predetermined periods of time, into research groups interested in this
experience. The goal is to combine their different expertise. These collab-
orations have had various goals, including the following examples:
providing the musical evaluation of computer tools, making exam-
ples, or writing user documentation;
- - conducting shared thought and discussion on particular preoccupations;
- - as pre-production for a musical project which calls for the collabora-
tion of scientists, from the earliest stages of the project - - so as to
choose the right direction from the start and to gather as many
resources and as much documentation as possible.
As a result, n e w relations are created between them; although their
motives might seem different. For the composers, it is important to have
permanent and complete access to the representation, manipulation and
organization of sonic material, as well as its parametrization and fabrica-
tion. For the scientists, the goal is to achieve a better understanding of
phenomena linked to the production and perception of sound, and to
find systems for accessing and controlling this knowledge. If the com-
posers use the scientific tools for their own (selfish) needs, they also vali-
date the research by providing it with a concrete reality and by opening
its use to very varied applications. If the researchers profit from the
musical expertise of and propositions made b y the composers, they also
pay attention to their concerns and regularly provide them with tools
(and improvements for their tools) coming from their most recent work.
This is somewhat reminiscent of the role held by acoustical instrument
makers. The composers are introduced to the technical domains of digital
audio and cognitive sciences, while the researchers acquire knowledge
about the art of music and its associated issues.
The multidisciplinary field of research has, thus, been enlarged these
last few years, b y drawing upon not just computers, acoustics, synthesis
and signal processing, but also psychoacoustics and the necessary cre-

3. translator's note: This term translates literally as composer in researcK In practice, the
position corresponds to that of a compositional consultant to the research groups.
 Technology and Creation 65

ation of adequate tools for musical formalization. Faced with composers

insistence and thanks to their persuasion, the research groups have
started paying more attention to, the often neglected aspects of, user
interfaces. The elaboration of these interfaces requires time, as well as
very specific qualities and know-how. IRCAM allocated the necessary
means for this kind of development and called upon a collaboration
between composers, musical assistants and software developers. In
effect, the conception of a user interface has a decisive influence on the
composer's ability to develop an artistic expression, formalize ideas and
share knowledge.

From sound to model

Composers, working with a computer, manipulate sonic events which

have already been coded and standardized. They are, however, also
involved in an interactive, experimental process where the very nature of
the sounds interacts with their work. This has led them to develop
musical ideas, not only through traditional means of notation and
general compositional technique, but also through observation - - the
study and conceptualization of these ideas within a computer environ-
ment. It is often necessary to wait for the result of a calculation, an evalu-
ation or a sonic synthesis, in order to concretize and validate an idea.
This is the reason that an auditory control at each phase of the composi-
tion has gradually become indispensable. Going even farther, we might
even suggest that the software environment has become central to these
composers' efforts to find new ideas, experiment with them and organize
them into musical compositions.
This environment must allow composers, by themselves, to construct
the elements which will be used throughout the realization of a piece; to
proceed into the phases of calculation and organization of the musical
material without external help, relying essentially on their own knowl-
edge and experience. While activities like synthesis and composition
have always been separated by an historic heritage, at the beginning
caused mainly by the deficit of necessary computing power and then by
the difficulties in reuniting these two aspects, composers and researchers
have recently tried to integrate these tasks in a shared environment, or to
make communication simple (this has been partially due to the progress
made in object-oriented programming).
Figure 1 presents a flow chart showing an overview of the relations
that can exist between the different tasks performed in a musical context.
The arrows indicate the links or interactions between the different, some-
times simultaneous, functions:
66 EricDaubresse and GdrardAssayag

CI~MIKI~ITION 4 9 PB:F-G~ANCE
coonception & f ormalisat ion- -apparatus for producingand
treatments & algorithms & spatializing - score following &
rules- interact ion wit h inatrurnents - real-
experimentation time control systems

INTERFACE 1L

I
EDITINGAND SYMBOLIZING
~,
multi- dimensional
represent atlons:musi
cal score, analysis /
J
J

editing sounds, cent rol - " algorit hms-signal

I parameters and musical J processing& construction of virt ual
structures; sequences, lists of
J instruments -synthesis control -
notes and values, manipulation
r 9 J generation and organizationof sonic
Jof models material

Figure I

Many attempts have been made to reduce the contradictions between

the representation, in musical language, of a musical problem and the
explicit rational methods used by an engineer while creating a user inter-
face. One of the first functions offered by the computer was the possibil-
ity of modeling natural phenomena. (As we have already emphasized,
analysis, artificial reproduction and experimentation of and with these
models has provoked a series of interdisciplinary research projects that
have led to results which were immediately used b y the participating
composers.) These models are often expressed either on axes of fre-
quency and time or frequency and amplitude; they can be assimilated
into a form of abstraction containing, in a formalized way, the informa-
tion concerning the sound, along with information about the apparatus
for reproducing and controlling that sound. This makes it possible to
separate different aspects of the model and more subtly modify or extract
particular characteristics.
The representation is one of the first aspects of the concept of a model;
it remains central to the understanding of the object, even if it is often
limited to one or two of its dimensions and is sometimes difficult to
analyze for the non-connoisseur. However, this is not without precedent,
since a musical score is also a reduction of parameters - - in that case,
interpretation enriches the representation while remaining coherent to
the model. The essential aspect, thus, lies in the manipulation and inter-
pretation of this representation: if acoustic instruments have a mode of
access which allows a large variety of interpretation, computer models
 Technology and Creation 67

are much less easily manipulated (this is due to the impossibility of

obtaining an access that is both complete and 'instrumental,' or to the
parameters necessary to completely describe instrumental behavior,
which would be too numerous to provide). Nonetheless, sooner or later
this representation must move into the symbolic domain, which will
allow the composer to partially reduce the amount of analysis data,
thereby facilitating the manipulation of the musical material with the
proper musical tools. He will, however, also act through other represen-
tation systems on the parameters generated by the models, performing
other manipulations on the sonic material or its representation: for
example, mixing the sounds, making a hybrid of them, transforming
them and even creating new (see note 2 on page 61) timbres. It should be
noted that while the field of investigation is vast, while the preoccupa-
tions are both musical and scientific, we are witnessing, more and more,
a redefinition of the concept of a model, in its structure and its manipula-
tion. The multi-representation of these models allows them to be
approached from different angles: for example, as intervals, a spectrum,
a relation between a fundamental and partials, etc. From conceptual rep-
resentation to semantic fabrics, the conditions are now present for the
manipulation of musical material and the generation of forms or
processes.

Computer assisted composition

Computer Assisted Composition (CAC) has headed towards becoming

an independent discipline for the last several years. Spectral composers
have made important contributions to this evolution, and it would be
worthwhile to spend some time investigating what is included within
this denomination.
The use of computers for music is mainly known to the public through
synthesis and sonic treatments. However, things did really not begin that
way: the first experiments, notably performed by Hiller, Xenakis,
Barbaud, were purely formal and combinatorial. The computer was
already being used in its capacity for treating symbols and relations (like,
for example, those defined in harmony treatises). This first approach to
symbolic computing was not successful: effectively, the simple applica-
tion of rules, especially restrictive ones, does not guarantee that the
results will be interesting. From a purely theoretical perspective, this can
be explained by the fact that the combinatorial space being searched for
solutions is gigantic. The zone of significant musical results within that
space, however, is not defined by any criteria. The chance of ending up
68 Eric Daubresse and G&ard Assayag

there after an exploratory process is infinitesimal. Xenakis, after having

formulated a still famous critique of the combinatorial approach, moved
towards a statistical approach - - implying a fundamentally different
mathematical vision and leading to a singular style. Simultaneously, the
techniques of digital synthesis began a revolution which would for many
years eclipse the symbolic/formal approach.
This approach would again flourish in the 1980's; as the result of several
converging causes. The first was purely technologic and was linked to the
evolution towards personal computer systems, with more and more intu-
itive user interfaces. The result was that large numbers of composers
could make use of these tools for experimentation, without depending on
the cumbersome and opaque technological infrastructures which could
only be used by people with the appropriate scientific and technical back-
ground. The other causes are of a more aesthetic nature. The first experi-
menters in computer music approached the problem from an angle
similar to the mechanistic ideas of the eighteenth century. The composers'
work was to create an automaton which would then, as a second step,
create the musical work through the workings of its internal mechanisms.
Thus, they needed a synthetic vision, although fictive, of musical
processes for which harmony treaties constitute the ideal paradigm.
The following generation took note of the great evolutions in musical
research between the 1950's and the 1970's, especially the emergence of
formalized material as a central category in the compositional process.
Hugues Dufourt described what he perceives as an antinomy between
the categories of pitch and time in the music of Boulez and Stockhausen:
organized series, symmetrical constructions, harmonic operations
like chord multiplication for Boulez; event density per time unit,
classification of types of attack and articulation, quantification of texture
changes and timbre for Stockhausen. Rationalist, constructivist, non-
synthetic, this approach is perfectly adaptable to numerical and formal
calculation - - accessible thanks to the computer.
In the context of the 1970's, while an aesthetic already on the decline
nonetheless furnished the bases for a calculationally accessible rational~
ity and while the power of synthesis and treatment technologies
increased, along with analysis of sonic phenomena seen as timbral cate-
gories, the spectral composers appeared as the catalyst which would con-
siderably aid the emergence of CAC. Though they constructed their
language on a new foundation (a language which, Dufourt has said, is
marked by transitions and an art of duration and development), they
accepted this rationalism of material, already espoused by Messiaen,
which can be described and therefore constructed, at least partially, by
objective procedures - - free from subjective content. The reference to the
acoustic functions of the sonic world, whether concrete sounds, instru-
 Technology and Creation 69

mental sounds or synthetic sounds, creates an innovative bridge between

formal constructivism and the emancipation of timbre: previous states
within the evolution of musical thought, had, most often, characterized
that relation in an antinomic manner.
The spectral musicians created a real need for computer technology
and often took the lead, impatient to obtain tools that lived up to their
experimental imagination. From the early 1980's, Tristan Murail devel-
oped programs on the first commercial micro-computers. On these rudi-
mentary machines, he was already able to calculate harmonic processes
derived from synthesis procedures and electro-acoustic treatments, like
frequency and ring modulation. This harmonic material could be visual-
ized as musical notation, then manipulated before being organized into
higher level structures. Murail also used the data from spectral analyses
and combined the analysis data with simulated synthesis: the opening
section of Gondwana (1980) is a harmonic interpolation between a chord
constructed by frequency modulation (simulating a bell sound) and a
trombone spectrum whose frequencial components were derived
through a Fourier transform.
Claudy Malherbe and G6rard Assayag, in the musical research depart-
ment of IRCAM, introduced a CAC environment which allowed the
definition of calculations effecting various musical parameters and
printed out the results in standard musical notation. They used the
psychoacoustic algorithms of Terhardt to analyze complex sounds, like
multiphonics or bell sounds; limiting the mass of data provided by
the Fourier transform to only the perceptually pertinent components.
Claudy Malherbe used this system to perform an instrumental synthesis
in which a group of woodwinds recreate, through orchestration, the
multiphonic sounds produced by these instruments.
In Modulations (1978), G6rard Grisey used the acoustic phenomena of
differential sounds. The generative sounds are in the organ and in the
brass. The differentials are played by the rest of the orchestra. Grisey
states that such a network of additional frequencies, a typical example of
structured, calculable material, functions like the shadows cast by the
main pitches; thus creating a new space for orchestral timbres in an
orchestration controlled by the pitches.
In Paradigms (1993), Joshua Fineberg advances instrumental synthesis
another step forward. Complex, previously composed, instrumental tex-
tures which have been played and recorded are analyzed in order to
deduce the harmonic material which is, subsequently, used to compose
the final piece. In this way various acoustic effects, such as interference,
which appear only in live performance, are taken into account.
In all of these examples, calculation is indispensable. Whether or not
this calculation is performed with the help of a computer, whether or not
70 EricDaubresse and G~rardAssayag

it is prolonged to higher level procedures, effecting the transition from

the material (which is a reservoir of potentials) to the discourse (actual-
ization of these potentials) depends on each composer. However, the
description of the basic objects and their relations as calculable functions,
the reference to acoustic functions whether as sensory information or as
models for the generation of material, is firmly etched in recent musical
history.
Since the late 1980's a marked acceleration has taken place in the field
of CAC. Powerful programs, such as PatchWork (Laurson, Duthen,
Rueda) and OpenMusic (Assayag, Agon), have appeared. Strongly
influenced in their conception by spectral composers (to whom one must
add musicians like Magnus Lindberg) and characterized by a relatively
recent concept of computing, visual programming, they have allowed a
large number of musicians to learn to use computers in a relatively pain-
less way. These programs allow composers to work very interactively
providing visual and auditory control at all steps in the calculation of the
material.
They have, notably, allowed the refinement of an important musical
aspect, revealed by the spectral approach: if acoustic models, through
computer treatment, are to furnish the elements of a musical palette, the
question of how they should be transcribed in the written code of
musical scores still requires resolution. Where the question is transcrib-
ing fixed pitches, the problem is not very difficult. However, once the
musical structures take on more breadth and especially when they imply
an 'art of development and duration,' the difficulties can increase pre-
cipitously. To be clear, how can a dynamic sonic process, in which multi-
ple parameters evolve in time, be transcribed in discreet units and
symbols compatible with instrumental notation? Progress, in this
dilemma, has recently been accomplished thanks to the combination of
signal treatment tools, based on the phase vocoder technique (analysis in
time/frequency) and the tools of CAC. These latter can make use of the
time/frequency representations to deduce a symbolic form including
meter, rhythm and harmony. Thus, the scales at which materials can be
calculated by the computer is increased. Starting with fragmentary ele-
ments of harmonic vocabulary, it has progressively attained the capacity
of generating complete musical processes.
As we will see below, Claudy Malherbe used, for his piece Locus
(1997), the program AudioSculpt for time/frequency analysis and
PatchWork for transcribing pitches and rhythms. He was able, in this
way, to deduce all the harmonic and rhythmic material from a body of
recordings of spoken sounds - - whose prosody, vocal colors, consonants
and pacing function as generators for the musical processes.
 Technology and Creation 71

Case studies

Four composers w h o have worked, in recent years, at IRCAM have real-

ized pieces which share, despite their aesthetic differences, a common
sensitivity toward the notion of models. These composers have all used
the computer and have reflected on the innovative aspects of using com-
puter technology. However, most of all they have all used this technol-
ogy in a musical way, thus providing it with a precious validation.
They are:
Joshua Fineberg for his piece Empreintes
Tristan Murail for his piece l'Esprit des dunes
Philippe Hurel for his piece Lefon de choses
Claudy Malherbe for his piece Locus

Empreintes by JoshuaFineberg

In this mixed piece which associates electronics with instrumental

writing, Joshua Fineberg wanted to explore new territories within the
field we call interaction: in other words, the relation between the instru-
ments and the computer. The idea behind this approach is to find a
retroactive common ground where the electronic emissions and the
instrumental playing can meet. Works of this type are generally divided
into two categories, both with their limitations and neither of which is
completely satisfying.
The first uses real-time in a too schematic way, capturing the instru-
mental sound, finding its virtual fundamental and sometimes its ampli-
tude envelope, then converting them into numerical parameters; these
are then compared to the score, already written into the computer, by a
'score-follower.' This technique allows the precise synchronization of
electronic events in relation to instrumental playing, but has the draw-
back of reducing the music to a few elementary values, like notes and
velocities. The attributes of timbre, color and movement are eliminated
so as not to use up too much computing power. Another important
drawback to this technique rests in the impossibility of polyphony; since
there are no sufficiently reliable algorithms for separating polyphonic
voices, and those being used experimentally demand too much comput-
ing power for real-time. The result of this technical limitation has been
the generation of pieces written for a single (monophonic) instrument or
for a soloist within an ensemble; this choice being made not for musical
reasons, but for technological ones.
72 EricDaubresse and G&ard Assayag

The second category of mixed works gives free reign to compositional

imagination, since the calculational limitations of real-time are elim-
inated. The phases of analysis and generation of material are performed
outside of real-time, then recorded and only their playback is triggered
in concert. This type of approach has allowed the realization of pieces
with impressively rich electronics and with compositional projects built
upon very elaborate manipulations; however, the synchronization with
the orchestra (or ensemble) remains a problem as is the use of the fine
instrumental variations from performance to performance in the control
of synthesis parameters. The analysis operations are mainly conducted
on a relatively simple and harmonic material, leaving out continuous
variations or complex orchestral characteristics like brilliance or the beat-
ings within timbres or harmonies.
The solution which was envisioned, to combine the advantages of both
of the methods described above, already partially existed. A computer
toot, iana (instrument analysis), implemented by Dan Timis and G6rard
Assayag and following the principles for extracting pitch salience from
the algorithm of Ernst Terhardt. 4 had already been in use for several
years at IRCAM in a non real-time version. This algorithm had immedi-
ately garnered the interest of a number of composers: due, notably, to its
ease of use and the pertinence of the results it generates. Originally con-
ceived for psychoacoustic analyses, the use of this tool has turned out to
be very interesting from a musical perspective - - especially for resynthe-
sis. Joshua Fineberg's musical project required the real-time implementa-
tion of iana; both to analyze the sonic material coming from the
instrumental ensemble and to synchronize the ensemble with the elec-
tronic apparatus.
Todor Todoroff, from the Royal Conservatory of Mons (Belgium),
reprogrammed the algorithm in the environment MAX-FTS (running on
the IRCAM Signal Processing Work Station) in such a way that com-
posers would obtain identical results in real-time to those of the older
non-real-time implementation. The only difference being in the round off
errors (the formats for storing real numbers are different in the two ver-
sions), which do not significantly alter the results. Iana~ (the tilde
identifies the real-time version) generates lists of the component frequen-
cies found in the spectrum of an instrument or a group of instruments
analyzed. These can be classified by perceptual criteria, so as to give
composers pertinent information about their respective importance. The
data is output from iana~ in the form of a series of lists. Each list corre-

4. Terhardt Ernst, Stoll Gerhard & SeewannManfred (1982) 'Algorithm for extractionof
pitch and pitch salience from complex tonal signals,' ]ASA 71 (3), March 1982,
pp. 679-688.
 Technology and Creation 73

sponds to a frequency component and is made up of four numbers: the

peak's ranking (its number), its perceptive weight, its frequency, and its
amplitude. The user can choose for all peaks to be transmitted or only
those with a positive perceptual weight (the only ones which contribute
to the spectral pitch pattern). Working in real-time allows the creation of a
stronger link between the ensemble and the treatments performed by the
computer; since it makes possible the continuous extraction, in a concert
situation, of information about the spectra generated by the instruments
of the ensemble. Certain musical processes, which previously had to be
calculated in advance, can n o w truly depend on the instrumental
playing.
All of this takes place on the IRCAM Signal Processing Workstation
(ISPW), a system which consists of a NeXT computer and three signal
processing cards, equipped with AD-DA converters (piggy boards). The
audio connections of the cards, for this piece, with the exterior are the
following: one input (sounds to be analyzed) and six outputs (for
the spatialized synthesis).The microphones which capture the sounds of
the ensemble's 14 instruments are pre-amplified and sent to a 'switch'
type matrix (at the premiere, this matrix was realized by a Yamaha DMC-
1000 digital mixing console, controlled by the ISPW) so that a selective
choice of the instruments can be sent to the iana- module for analysis. It
is possible, in this way, to address the whole ensemble, a sub-grouping of
the ensemble or even a single instrument. The station's input is passed
through a dynamic filter, made from two band-pass filters, which can cut
out or amplify bands of frequencies, to eliminate or favor certain zones
of the spectra according to the desired type of result. The analysis
module, placed on a single of the station's six microprocessors, is sur-
rounded by three banks of 14 oscillators, 42 total; 6 of which can also
receive phase modulation. A bank of 12 second order filters was also
built; it can receive its excitation source from three internal samplers. The
creation of this 'patch' and many other aspects of the apparatus used for
the piece were created by the composer in close collaboration with one of
us, Eric Daubresse.
Figure 2 shows a synopsis of the signal (analog and digital) connec-
tions along with the MIDI connections.
The interesting idea behind this implementation is to be able to analyze
complex sonic phenomena while still obtaining a result with a relatively
small number of values (which are the most pertinent). It is this idea of
sonic 'empreintes' [imprints], which gives the work its title. The synthesis
results depend, of course, on the instrumental playing, but also on the
w a y in which the programming changes the characteristics of the data
extraction, since they are all controllable in real-time. In this w a y it is pos-
sible, for example, to analyze a single instrument, focusing the detection
74 Eric Daubresse and G&ardAssayag

MIDI K E Y B O A R D KX 88 ~ sustainpedal

I midi

MICROPHONES

V
NeXT
PREMAMPLIFICATION ~ ADC with3 cards
11,o4,oo
I comp~sor [ DACS

] 14 ADC 5.'~C

D M C 1000-1 D M C 1000-2

[ 1DAC I I 8DACS

I
TOWARDTHE8 SPEAKERS
Figure 2

on dynamic differences or the timbre of the phrasing. Operations on the

lists (filtering, interpolations, modulations, etc.) which modify their con-
tents are also performed directly on these analyzed models rather than on
the sound itself, thus reducing the cost in calculation.

L'Esprit des dunes by Tristan Murail

Also realized at IRCAM, this piece raises issues which cannot be reduced
to the simple opposition real-time or non-real-time. It attempts to con-
struct mechanisms for moving from concrete sounds toward abstract
models; to appropriate musical elements s o as to better understand and
manipulate them; in sum, to allow the passage from raw musical models
toward elaborate instrumental ones. The composition was begun by first
making use of analysis fragments from different sources, including
diphonic Mongolian singing, Tibetan singing and trumps along with
Jew's harps.
 Technology and Creation 75

All of the piece's material, instrumental and synthetic, was generated

from these spectral analyses. The methodology is central to the elabor-
ation of this piece. Work was conducted simultaneously in the studios at
IRCAM, for operations requiring large amounts of computing power,
and in the composer's personal environment, for the rest.
The composer's need for a finer time-frequency representation was
resolved through the use of a recent, extremely elaborate, analysis tool
created for additive synthesis. The calculational kernel of this tool was
developed by Guillermo Garcia, within the analysis-synthesis research
group at IRCAM. In effect, of all the techniques for generating sound,
additive synthesis holds a special place by virtue of its reconstitution of
sounds by their elementary components called partials and modeled by
dynamic oscillators. Besides its rapidity, the algorithm allows the resid-
ual noise of sounds to be separated and isolated from the detected par-
tials. The trajectories of the partials remains coherent, most of the time,
thanks to a screening of the data and a spectral modeling technique. It is
possible to limit the number of components analyzed, so as to control
the resynthesis without too great a cost in computational power.
Work on the piece was divided into four phases:
Analysis of the materials, estimation of the parameters for the additive
analysis program and, once a first real-time evaluation of the quality
and interest of the selected fragments has been performed, the con-
struction of a database of analyses for the piece (the evaluation was
based on the objectivity of the analyses and their musical potential).
-- Evaluation and modification of these analyses with a library of tools
created by the composer, with some help from IRCAM's musical rep-
resentation group, in the CAC program PatchWork (developed at
IRCAM by Laurson, Assayag, Duthen, Rueda, Agon ...)
Among the main tools used by Murail, can be noted:
- - filtering one or more components of a spectrum by reducing or
increasing its amplitude
- - fractionating a spectral region, which is removed
- - smoothing the frequencies or dynamics of components
- - t e m p o r a l modifications of the spectra, by displacement or de-
synchronization of some of its components.
- - First sketches, both compositional and for the control of the computer
environment; then final writing of the musical and computer scores.
-- Final synthesis and mixing, with an aural control of the results and
final adjustments 'by ear.'
The composition of the piece took place simultaneously with the work
in the studio. The back and forth between analyses, evaluations and com-
76 Eric Daubresse and GdrardAssayag

position were constant and caused modifications at each phase in the

conception of the work. For this reason, it was essential to be able to have
a quick and constant aural check of the results; this allowed the eventual
modification of the values given to the various parameters to be as
intuitive as possible, while nonetheless allowing great precision.
Serge Lemouton implemented an interface for controlling the analysis
parameters and an additive oscillator or second order filter type synthe-
sizer in MAX-FTS that works in real-time. This synthesizer also has com-
plete modularity in terms of the functionality related to the synthesis
apparatus. It can memorize all of the settings related to a given situation,
visualize and refresh the parameters, and go from a bank of filters to a
bank of oscillators without restarting the application. In its complete
version, the synthesizer allows for the control of 40 filters, which makes
for a comfortable quality/cost compromise, with very good quality.
The main synthesis controls accessible from the interface are:
- - the amplitudes for each partial
- - the frequencies for each partial
- distortion of the frequencies
-

- temporal manipulations, stretching or compression

-

- - bandwidth for the filters, when the resynthesis is with a bank of filters
- stereo spatialization for each partial
-

- - delay in the appearance (start time) of each partial

These controls allow the rapid manipulation of the parameters of each
sound, through global MAX messages, which are sent as continuous data
streams or as discreet floating point values.

LeFon de choses by Philippe Hurel

In this piece for ensemble and electronics, Philippe Hurel worked with
the ideas of simulation and contamination. Simulation offered an inter-
esting paradigm for musical creation; since, starting from an established
model, different types of variations, affiliated with that generic model,
are generated. Contamination, whose principle came from a book by
Claude Simon with the same title as this piece, proceeds through various
operations of contagion between two or more simulations. Initially, con-
crete sonic objects were analyzed and two of them were chosen, because
of their color, spectral richness and potential for manipulation: a pie-tin
and a tape-reel struck with a drum stick. These sonic objects were ana-
lyzed using the principle of resonance models. In other words, FFTs were
calculated with several windows of different lengths, the peaks of each
analysis were extracted and then compared to find the invariants. From
 Technology and Creation 77

this comparison a spectral envelope is extracted, modeled in the form of

a source-filter type synthesizer (that is to say, one comprised of fre-
quency, amplitude and b a n d w i d t h values) which represents the reso-
nance of the instrument and can control filter banks. The processes of
discretization are realized with parameters that the composer adjusts and
which he, like Murail, systematically subjects to aural evaluation.
In a second phase one of us, Eric Daubresse, constructed in MAX-FTS
an environment that allowed the simulation of the mode of sonic pro-
duction of the sonic object through the implementation of second order
resonant filters. These filters, which could create a satisfying simulation
of the original sonic object, could also be completely controlled in real-
time. A sequencer allowed each of the 64 filters to be controlled, by
sending them at a given time messages telling them to open or close that
would then be realized by their dynamic envelopes. The manipulation,
thus, is mainly temporal, but can also effect:
- - the pitches, b y receiving messages describing new spectra derived
from the original
- - the dynamics of each component, through the envelopes, but also
through the MIDI control of each volume
- the bandwidths, by effecting the relation between the bandwidth and
-

the band-amplitude
- the timbre, since it is possible to filter (by what is called the resonator)
-

a source (called the excitation) other than white noise, which is

considered neutral.
Thus, one can find here the idea of contamination, which takes on its
full force in the manipulation of objects derived from the original.
Crossings, hybrids, and deviations have, thereby, formed the essential
part of this work. The division between the modes of excitation and reso-
nance allowed the powerful constituents of the models to be separated
into two different entities which can be manipulated independently. One
of the most interesting characteristics concerning this manipulation con-
cerns the durations; timbral interpolations of varying durations can be
performed and can therefore effect the perception of the successions. The
most important is the trajectory between two states and not the starting
and ending states themselves. The opposition between the simulation of
objects and their deviations-evolutions served as compositional genera-
tor for the piece: the concept of patterns being central to the structural
and metric organization (editor's note: cf. the article by Damien Pousset,
in the second issue).
The composer used the analyses and their manipulation not just to
produce the synthesis, but also to generate the instrumental material.
One, effectively, finds the same objects as in the synthesis, undergoing
78 EricDaubresse and Gdrard Assayag

identical manipulations and maintaining the same relations with the syn-
thesis as exist between the objects themselves. This is supplemented by
the interactions excitor-excitee and situations of fusion b e t w e e n the
ensemble and the synthesis.
A direct application coming out of psychoacoustic research is the use
of auditory stream theory in this piece. During the second section,
sampled instruments take over for the real ones and due to the linked
usage of the pitches played and the speed at which they are played, the
listener perceives a multiplication in the number of instruments and an
enrichment of their spectra. This effect is even more pronounced because
of the simultaneous spatialization between these instruments and an
interpolation of their timbres.

Locus by Claudy Malherbe

Synthesis is an especially demanding experience which will not allow for

chance, especially when confronted with the human voice, whose rules
of production are particularly complex. If speech synthesis has under-
gone a rapid expansion, its musical use has faced great difficulties.
This piece for four voices, premiered by the English ensemble Electric
Phoenix, is built on a transfer from the world of real sounds towards a
musical resolution. A discreet representation of flowing speech, some-
times noisy, sometimes sung, is provided by analysis. Manipulations
using symbolic calculations are then performed on these representations.
As we have stated above, the recent musical and technological evolu-
tions have helped bridge the gap between previously separate worlds:
w o r k on complex sonic material n o w maintains a tight relation with
symbolic manipulation, allowing these two approaches to mutually
influence each other. Thanks to the integration of various functions
within the same program or through an easy communication between
applications, it is possible to obtain a multi-dimensional physical repre-
sentation of a complex sonic reality: to convert it, interpret it, and ma-
nipulate it in a coherent manner. The different phases of recording the
material, analyzing it, performing calculations on it as well as its passage
towards the symbolic world and its final mutation into music, were occa-
sions for a profound reflection on the methodologies of this itinerary of
deconstruction-construction and on the articulation of its different steps:

1 Generic sonic object, recordings:

After some test recordings on the voices of the singers, a certain number
of criteria were selected to offer as faithful and rich a material as poss-
ible. First is the use of two microphones, with distinct characteristics,
 Technology and Creation 79

for the recordings: one extremely close to the mouth and the other at a
m e d i u m distance. Then there was the need for a constant attention to the
quality of interpretation and the number of takes. And finally the selec-
tion from these takes, by successive approach, at the moment of analysis.
Segmentation, cutting into pieces, smoothing and normalizing the mater-
ial are also necessary; especially in the final phase of articulation, when
these sounds will be orchestrated.

2 Time/frequency analysis:
The result of an analysis by FFT, using parameters freely defined by the
composer and essential to future operations (window size, window step,
number of points, etc.), this type of analysis allows a first representation
of the musical material, in the form of a sonogram. It was then necessary
to perform a linear segmentation of the sonogram, according to physical
(energy), phonological (text) or musical criteria, before proceeding to the
detection of partials. The most pertinent of those partials were then
selected and a smoothing of the frequencies and amplitudes was per-
formed on them (to reduce the amount of data).

3 Symbolic conversion:
The data resulting from the analyses are then exported to the program
PatchWork where, after a second phase of data reduction (micro-interval-
lic approximation, polyphonic density, range, harmonic reduction) they
are transcribed and submitted to symbolic manipulations.

4 Composition and articulations:

The initial rhythmic representation is in proportional notation. A rhyth-
mic editor (Kants) allows the re-dimensioning of the material's metrical
content, through a heuristic approach which leaves the composer with
the choice between multiple propositions for the segmentation and the
tempo. On the harmonic level, mutations are applied to the original
material. These changes conserve the basic color while still proposing
variants. This is accomplished, notably, by changing the fundamentals,
through the auto-transposition of a chord onto itself or through an
algorithm which transforms a harmonic sequence by substituting the
chords with other chords, drawn from a pre-determined reservoir using
a principle of minimal distance.
Thus, the results of these transformations are still correlated with the
original objects and can, in the final realization, be recombined with

5. Developedat IRCAMby the Computer Assisted Composition group (Agon,Assayag,

et al.).
80 EricDaubresse and G~rardAssayag

them - - closing the loop. In this way, the final object is formed b y a
dense fabric in which all the steps in the elaboration feed back on one-
another and create articulations between them.

Conclusion

It is indisputable that the use of new technologies have allowed the dis-
covery of previously unexplored territories. Composition and synthesis
are becoming, bit by bit, one and the same thing, and are, heretofore,
linked by the desire of composers to integrate and control all the phases
of composition. Electronics are manipulated like an orchestra, notably
through their multi-representations, and the orchestra is treated like a
model whose realization is performed by instruments.
One of the underlying goals is to elaborate models which give a satis-
fying representation and provide for pertinent manipulations.
Nonetheless, there is not always a one-to-one correspondence between
the physical parameters used b y the computer and the perception of the
listeners. It is the composer's role to integrate the mechanisms of com-
pensation and interaction. This is the reason that a large element of
choice is left to the composer, so as to be able to intervene at any
moment; to chose one direction rather than another, in function of aes-
thetic criteria and not scientific ones; to intervene freely, without any
special constraints, on the material.
There remain, however, vast fields that still lie fallow and solutions
still to be found. For example, systems that deal well with the micro- and
macro-control of synthesis, as well as their representation and connec-
tion to the traditional tools of the composer: in other words, systems for
high level control.
However, the know-how that already exists must not be lost. It is also
important that quick and efficient solutions for porting pieces from one
platform to another be found, along with a reflection on solutions which
w o u l d create a true independence of platform. The spectral current has
brought a certain number of changes in the w a y that computers are
approached and the tools that the composers of this current have pro-
duced have turned out to have a much more general usefulness.