Professional Documents
Culture Documents
1 Introduction
This paper focuses on a specific compositional art named orchestration from
the perspective of Western classical music. Orchestration can be defined as the
art of combining pitches to compose music for an orchestra or more generally
an ensemble. This involves writing for a number of instruments and can be
seen as a symbolic view of composing. Furthermore, orchestration is the art of
mixing instrumental properties. For example, by combining small sounds from
different instruments, the orchestration creates a sound that could not exist on
its own. This second aspect can be described as the sonic view of composing.
Orchestration is an interesting compositional art as it can help achieve a musical
idea that cannot be done with a single instrument. Even if some attempts to
define the art of orchestration have been done, such as in [7], [19], [20] and
[1], its teaching and practice are mainly empirical. This is because there is no
mathematical foundation or long theoretical traditions for this activity as there
is for other compositional aspects. This is possibly the reason that this musical
discipline is said to be “at the crossing of daring and experience”[24].
Since the invention of the computer, composers have been interested in ex-
ploring its potential in the compositional process. The Illiac Suite [16] was one
of the first pieces to use computers to compose music. Since, several different
computing techniques and systems have been developed for musical composi-
tion. These tools mainly allow composers to manipulate musical symbolic ob-
jects, such as notes or chords giving them the ability to focus on harmony and
2 Aurélien Antoine, Eduardo R. Miranda
2 Computer-Assisted Orchestration
In his piece L’Ésprit des dunes (1994) realised at IRCAM, Tristan Murail
started his composition by analysing fragments from different sources, such as
diphonic Mongolian singing and Tibetan singing. The material of this composi-
tion, for an ensemble of 11 instruments and electronics, was generated by spec-
tral analysis of the aforementioned sources. He used an analysis program devel-
oped for additive synthesis, then constructed a database of these analyses to be
evaluated and modified with libraries he developed in the visual programming
environment PatchWork [3].
Some composers used speech analysis for their compositions. Clarence Barlow
developed a technique called Synthrumentation, which consisted of doing spec-
tral analysis of speech and then mapping these analyses to acoustic instruments
[5] [27]. Claudy Malherbe also used voice analysis techniques for some of his com-
positions. In Locus (1997) [23], a piece for four voices and electronics, Malherbe
recorded the singers using two microphones placed at two different distances
in order to have two different characteristic recordings. After the segmentation,
smoothing and normalisation of the recordings, Malherbe applied a FFT anal-
ysis to obtain the representation of these recordings in the form of a sonogram.
Then, a detection of partials was applied and the most prominent were selected.
These data were subsequently input into PatchWork and transcribed into sym-
bolic representations for ease of manipulation. For the rhythmic representation
and manipulation, Malherbe used Kant, a rhythmic editor developed at IRCAM
by the Computer Assisted Composition group1 .
More recently, for his musical piece entitled Metal Extensions (2001), Yann
Maresz used a set of techniques combining handwriting and computational pro-
cesses. In [24], he described his process, as follows:
“Selection of the region of sound to orchestrate from the electronic sound file,
placement by hand of markers on the region within the sound file that interested
me, for a chord-sequence analysis with AudioSculpt (peaks), inharmonic partial
analysis on the totality of the sound file in the same programme, transcription
of the given results into symbolic notation in OpenMusic and finally, realization
of the final score by hand.”
1
http://www.ircam.fr/repmus.html
4 Aurélien Antoine, Eduardo R. Miranda
mentioned in the previous section of this paper. This resulted in three Ph.D. the-
ses [35] [8] [13] and computer-aided orchestration programs that evolved through
the years. Like the aforementioned systems, the user inputs a target sound and
the program computes an orchestration. The first version, named Orchidée, was
the result of the two Ph.D. theses written by Damien Tardieu [35] and Grégoire
Carpentier [8]. They respectively addressed the problem of analysis of instru-
mental sound and its perception, and the rapid increase of the possible solu-
tions produced by the system. The system extracts some audio descriptors from
the target sound, and also from the audio samples contained in the orchestra
database. These descriptors are the material for the combinatorial algorithm de-
veloped to match the target sounds [36][9]. The algorithm does not output only
the best solution but rather a selection optimal solutions, which is an advan-
tage as it proposes different orchestrations for one target sound. This version
does not consider temporal problems of orchestration. It proposes only static
orchestration solutions and the system works with static and harmonic target
sounds. Jonathan Harvey, assisted by researchers and computer music designers
from IRCAM, was one of the first composers to benefit from this new computer-
assisted orchestration system. He used it for his composition Speakings (2008),
which is for live electronics and a large orchestra [25]. He recorded three vow-
els, as mantra sung: Oh/Ah/Hum. Then, his idea was to input these recordings
into Orchidée in order to try to imitate the sound of the sung mantra with an
ensemble of 13 instruments.
The system evolved into a new version, under the name Ato-ms (for Abstract
Temporal Orchestration), and it is the result of the third Ph.D. thesis by Philippe
Esling [13]. One of the major improvements of this version is the management
of time, which generates orchestration solutions within a time space as opposed
to static. Another improvement is the use of a multi-objective and time-series
matching algorithm, which creates an optimal warping algorithm. In this version,
the user can design envelopes for the audio features thereby creating an abstract
target. According to Maresz [24], the solutions “suffer from a lack of quality in
their timbral accuracy” and the two versions only address the problem of timbre
matching.
In November 2014, IRCAM released a complete new version of this system,
named Orchids2 (Fig. 1). This standalone application implements the best fea-
tures from its predecessors and integrates new improvements. It proposes ab-
stract and temporal orchestration and is also optimised for timbral mixture.
Like the aforementioned systems, the user inputs a target sound, but in Orchids
they also have the ability to design an abstract target by shaping various psy-
choacoustic descriptors. Orchids also includes a database of over 30 orchestral
instruments, whose samples are analysed and indexed by the program. The user
can extend the sound database by simply adding a folder into the program, which
the system will analyse and index. The user has the ability to define the instru-
ments they want to include in the orchestra. Moreover, the user can position the
instruments, as Orchids integrates the notion of spatialisation of the orchestral
2
Orchids is available at http://www.forumnet.ircam.fr/product/orchids
6 Aurélien Antoine, Eduardo R. Miranda
space. The system first analyses the defined psychoacoustic features of the tar-
get. Different matching algorithms are available, specific to the type of solutions
the user wants or the more appropriate for the type of the audio file (see [14]).
Orchids usually proposes several orchestration solutions in the form of a musical
score. The program uses the Bach library3 for the symbolic representations. It is
also possible to listen to the solutions, thanks to the audio samples contained in
the database, before exporting the interesting orchestrations. Furthermore, the
user has the ability to start to construct and edit the composition directly inside
the program in addition to exporting it afterwards.
In this section, we have discussed three different approaches to try to incorpo-
rate the complex problem of orchestration into computer-assisted orchestration
systems. The latest program, Orchids, presents promising improvements for the
problem of computer orchestration and is set to be a powerful tool for computer-
aided orchestration. However, a problem we found in these systems is how to
classify the solutions or, in other words, how to guide the system to match the
kind of orchestration or sound we want. The aforementioned systems usually
propose several solutions, and the user can spend a lot time before the ‘best’
orchestration is found. In the next section of this paper, we discuss solutions of
these questions and introduce ideas to improve computer-assisted orchestration
systems.
3.1 Timbre
Working on computer-aided orchestration, we will focus only on instrumental
music, therefore in the attempt of defining the term timbre we will omit the
timbral characteristics of electroacoustic music. The notion of musical timbre
is complex and has been largely discussed in the last decades (see [21] or [34]
for examples). However, the American Standards Association [2] suggests the
following definition: “Timbre is that attribute of auditory sensation in terms of
which a listener can judge that two sounds similarly presented and having the
same loudness and pitch are dissimilar”. Furthermore, a note to the definition
adds: “Timbre depends primarily upon the spectrum of stimulus, but it also de-
pends upon the wave form, the sound pressure, and the frequency location of the
spectrum of the stimulus”. To summarise, timbre is all the sound properties that
enable us to distinguish and recognise one instrument’s sound from an other.
and Fc is a frequency. The highest the frequency is, the brighter the sound is.
Hence, we index the solutions, by their spectral centroid, from the brightest to
the least bright.
For the second timbral attribute, we chose to implement the perception of
roughness. This timbral quality is correlated with beats between two partials of
a sound, critical bands and partials above the 6th harmonic [15] [37] [4] [33].
We decided to use the mirroughness function from the MIRtoolbox v1.6.14 [22]
that integrates a set of functions written in Matlab5 . The mirrgouhness function
implements three methods of estimating the roughness of a sound. The first
method is based on an estimation proposed by Sethares [33]. The second method
is a variant of the Sethares model proposed by Weisser and Lartillot [39]. The
last method is also a variant of the Sethares model developed by Vassilakis [38].
4
MIRtoolbox is available at https://goo.gl/d61EO0
5
http://www.mathworks.com/products/matlab
Towards Intelligent Orchestration Systems 9
As per the brightness algorithm, we index the solutions from the roughest sound
to the least rough sound, based on their respective roughness value.
4 Final Remarks
5 Acknowledgments
This research is supported by an AHRC 3D3 Centre for Doctoral Training Ph.D.
studentship.
References
21. Krumhansl, C.L.: Why is musical timbre so hard to understand. Structure and
perception of electroacoustic sound and music 9, 43–53 (1989)
22. Lartillot, O.: MIRtoolbox 1.6.1 User’s Manual. Aalborg University, Denmark
(2014)
23. Malherbe, C.: The OM Composer’s Book, vol. 2, chap. Locus: rien n’aura eu lieu
que le lieu. Editions Delatour, France (2008)
24. Maresz, Y.: On computer-assisted orchestration. Contemporary Music Review
32(1), 99–109 (2013)
25. Nouno, G., Cont, A., Carpentier, G., Harvey, J.: Making an orchestra speak. In:
Sound and Music Computing (SMC). Porto, Portugal (2009)
26. Pennycook, B.W.: Computer-music interfaces: a survey. ACM Computing Surveys
(CSUR) 17(2), 267–289 (1985)
27. Poller, T.R.: Clarence barlow’s: Technique of ‘synthrumentation’ and its use in im
januar am nil. Tempo 69(271), 7–23 (January 2015)
28. Psenicka, D.: Sporch: An algorithm for orchestration based on spectral analyses
of recorded sounds. In: Proceedings of International Computer Music Conference
(ICMC). p. 184 (2003)
29. Roads, C.: The computer music tutorial. MIT press (1996)
30. Rose, F., Hetrick, J.: Spectral analysis as a ressource for contemporary orchestra-
tion technique. In: Proceedings of Conference on Interdisciplinary Musicology. vol.
2005 (2005)
31. Schubert, E., Wolfe, J.: Does timbral brightness scale with frequency and spectral
centroid? Acta acustica united with acustica 92(5), 820–825 (2006)
32. Schubert, E., Wolfe, J., Tarnopolsky, A.: Spectral centroid and timbre in complex,
multiple instrumental textures. In: Proceedings of the international conference on
music perception and cognition, North Western University, Illinois. pp. 112–116
(2004)
33. Sethares, W.A.: Tuning, timbre, spectrum, scale. Springer (1998)
34. Smalley, D.: Defining timbre—refining timbre. Contemporary Music Review 10(2),
35–48 (1994)
35. Tardieu, D.: Modèles d’instruments pour l’aide à l’orchestration. Ph.D. thesis, Uni-
versité Pierre et Marie Curie (UMPC) et IRCAM, Paris (2008)
36. Tardieu, D., Carpentier, G., Rodet, X.: Computer-aided orchestration based on
probabilistic instruments models and genetic exploration. In: Proceedings of Inter-
national Computer Music Conference, Copenhagen, Denmark (2007)
37. Terhardt, E.: On the perception of periodic sound fluctuations (roughness). Acta
Acustica united with Acustica 30(4), 201–213 (1974)
38. Vassilakis, P.N.: Perceptual and physical properties of amplitude fluctuation and
their musical significance. Ph.D. thesis, University Of California, Los Angeles
(2001)
39. Weisser, S., Lartillot, O.: Investigating non-western musical timbre: a need for joint
approaches. 3rd International Workshop on Folk Music Analysis (2013)
40. Williams, D.: Towards a Timbre Morpher. Ph.D. thesis, University of Surrey (2010)