Granular synthesis and spatialisation in the Pure Data environment
OSCAR PABLO DI LISCIA
Programa de Investigación “Sistemas Temporales y Síntesis Espacial en el Arte Sonoro”, Escuela
Universitaria de Artes, Universidad Nacional de Quilmes
Roque Saenz Peña 352
Bernal, Argentina, B1876BXD
odiliscia@unq.edu.ar
Abstract
This paper describes the design and some particular
applications in granular synthesis and spatialisation of a
Pure Data external developed by the author. It was
developed with the collaboration of Damián Anache and
Esteban Calcagno as a part of the research program
“Sistemas Temporales y Síntesis Espacial en el Arte
Sonoro” (UNQ, 2015-2019). The external offers all the
capabilities of GS synthesis with full 3D control of the
location of each grain through the Ambisonic technique.
The general approach of its design is to use audio signals
stored on a “pool” of tables as sources for both the grains
and the amplitude envelopes, allowing several shuffling
effects and synthesis/sequencing by concatenation.
Keywords
Sound Synthesis, Sound Spatialisation, Ambisonics,
Granular Synthesis.
1 Introduction
Generally speaking, Granular Synthesis (from here
on abbreviated as GS) is a technique based on the
juxtaposition of small portions (called grains) of a
source audio signal. In a typical GS application, the
user is allowed to set (and eventually change over
time) several parameters of a grain sequence, the most
common of which are: duration, temporal gap, source
audio signal, amplitude function, peak amplitude, and
pitch shifting.
There are many excellent GS computer applications,
either as standalone programs or as dedicated
modules, some of the most important being the GS
Opcodes for Csound (Specially Granule and
Partikkel, [2], [7], [16]), the GS units for
SuperCollider ([15], see also Del Campo 2011 [5])
and several external and abstractions for Pure Data
([11]). Of the latter, the most complete and powerful is
disis_munger~ ([3]), which is based on Dan
Trueman's munger~. Of the many GS standalone
programs available, one of the most powerful and
versatile is HourGlass ([17]). There are also many GS
software units in the form of Plug-ins (such as VST,
JSFX, or any other format) to be used on Digital
Audio Workstations.
GS can be refined in many different ways and
allows the production of very complex and rich
sonic events. Among the many interesting
features of GS, it is worth mentioning its
capability to produce smooth transformations of
individual sonic event sequences into swarms
of sounds (sonic objects) and textures or vice
versa. The use of GS on electro acoustic and
audio visual creation is very well known and
widely documented (see, among others, Roads,
2004 [13], Batty et al, 2013 [1], Del Campo,
2011 [5]), as well as electronic works by many
composers. The works by Barry Truax and
Horacio Vaggione are referred to in Roads’s
Microsound ([13] pp.235) as those which
explore extensively many of the possibilities of
GS synthesis since approximately 1970. Many
works by the composer Juan Pampin combine
GS and sound concatenation using high
resolution spectral analysis, audio descriptors,
and sound spatialisation most effectively, to
mention just one of the many recent
applications of this technique in sonic creation
(See, for example, a line (2015, [12]) ).
2 The my_grainer~ external
The my_grainer~ external1 offers all the
capabilities of GS synthesis with full 3D control
of the location of each grain through the
Ambisonic spatialisation technique (see below).
It was developed by Pablo Di Liscia with the
collaboration of Damián Anache and Esteban
Calcagno as a part of the research program
“Sistemas Temporales y Síntesis Espacial en el
Arte Sonoro” (Universidad Nacional de
Quilmes, 2015).
The general approach of my_grainer~ is to use
audio signals stored on a “pool” of tables
(maximal, 24 of them, at present) as source for
both the grains and their amplitude envelopes.
The user is responsible for the creation of the
tables, which can be done by either reading
1 http://puredata.info/Members/pdiliscia/grainer/gr
ainer.zip/view (Last access: 09/2016)
preexistent sound files, or using signals created “on-
the-fly” by synthesis methods or obtained from real-
time audio input. The strategy of using tables instead
of a signal input as the audio grain sources was
selected in order to allow shuffling effects using
several different audio sources. The user is allowed to
set the audio source and the amplitude envelope tables
for the next grain to be synthesized by means of a
message. A new message may either force the same
table to be overwritten with a different audio content
or to set a different table for the next audio grain to be
synthesized. There is also the possibility of alternating
different audio tables as audio sources and/or
amplitude envelopes out of the set of different table
names given to the external.
The choices for the grain duration, temporal gap, peak
amplitude, audio starting and ending read point on the
table, pitch transposing and/or warping, and spatial
location (azimuth angle, elevation angle and distance)
are handled by methods that allow the setting of a
base value and a random deviation. There is also the
possibility of setting the grain duration, temporal gap,
source audio table, starting read point and pitch
transposition by means of a random selection out of a
set of discrete numeric values stored in lists. The
probability distribution function of the random
selection is uniform, but non uniform biases may be
achieved including several repeated values in the list,
increasing the likelihood of their occurrence. If the
grain duration exceeds the time span determined by
the read and end points of the audio table, the user
may choose to use three types of loops (no loop,
forward loop or forward-backward loop).
The parameters for each grain (with the exception of
pitch warping) are invariant over its duration, which
means that any change will be taken into account for
the next grains to be synthesized. Most parameters
may be changed with a precise control for each one of
the grains. As an example of this, the time gap
between the grains can be set in five different ways:
a-Simply setting a base value in seconds without
any random deviation. This is usually referred to as
synchronous GS (in GS terminology).
b-Simply setting the range of a random deviation in
seconds and setting the base value to zero. This is
usually referred to as asynchronous GS (in GS
terminology).
c-A combination of the b and c possibilities will
produce a more controllable asynchronous GS.
d-Setting a list of discrete gap times which will be
chosen at random for each grain, to produce
asynchronous GS and to explore the potential of “time
fields”. In this case, the value previously set on a (if
any) will be disregarded. As this list can be changed
by successive messages, this can also prove useful in
order to produce gradual transformations from
sequences of individual sonic events with
statistically controlled time gaps on textures or
granulated sounds.
e-A combination of the b and d possibilities is
also possible, allowing random deviations of
the discrete gap values.
A comprehensive Tutorial on GS using
my_grainer~ is being developed by Damian
Anache2 and a Pure Data patch called
GRAINER_DINAMICS was developed by
Esteban Calcagno3 in order to allow the
automation of the parameters of my_grainer~
through editable functions stored in tables, so
that the users may store and retrieve very
complex simultaneous sequences of parameter
changes.
Finally, it should be noted that some features of
the external were also designed taking in
account its possible use on concatenative
synthesis. As noted by Diemo Schwarz
(Schwarz, 2005 [14]): GS can be seen as
rudimentarily datadriven, but there is no
analysis, the unit size is determined arbitrarily,
and the selection is limited to choosing the
position in one sound file. However, its concept
of exploring a sound interactively could be
combined with a pre-analysis of the data and
thus enriched by a targeted selection and the
resulting control over the output sound
characteristics, i.e. where to pick the grains
that satisfy the wanted sound characteristics.
As my_grainer~ can also receive and use a list
of pairs of starting times (in the audio table in
use) and durations for the grains, it may be
useful to achieve a sort of concatenative
synthesis as well. Each start/duration pair is
chosen at random out of the list, which could be
made up of data coming from a pre-analysis
and may even be updated by data coming from
real time audio input being analyzed.
3 Granular synthesis and spatialisation
using my_grainer~
my_grainer~ was specially developed to
experiment on GS spatialisation. This poses
some issues that should be taken into account.
If a non overlapping sequence of grains is
produced by a GS synthesis unit, each of the
grains could be individually processed by
spatialisation units that are independent of the
2 http://puredata.info/Members/pdiliscia/grainer/m
y_grainer_Tutorial_by_Damian_Anache.zip/vie
w (Last access: 09/2016)
3 http://www.mediafire.com/GRAINER_DINAMI
CS.zip (Last access: 09/2016)
GS unit (still, there is the problem of accurate
synchronization of each grain with a time-dependent
spatial location). However, if the grains overlaps, an
individual spatial location of each one is clearly not
feasible unless it is achieved from inside the GS unit.
This has the drawback that the spatialisation technique
used may not be freely chosen by the user. It is
feasible to produce a GS unit capable of performing
internally several individual spatialisation techniques
(such as Intensity Panning, VBAP, Ambisonics, WFS,
etc.), but some tradeoff between capabilities and
efficiency must be taken in account. That is the reason
for which my_grainer~ was developed to produce
mono, stereo (intensity panning), quad (Ambisonic B
Format First Order), nine channel (Ambisonic B
Format Second Order) or sixteen channel (Ambisonic
B Format Third Order) outputs. Ambisonics is a
technique that provides robust, versatile, and
standarized 3D spatialisation. It is widely used and
documented (See Gerzon [6] and Malham [10]).
Despite its perceptual effectiveness (which may
improved using high order encoding), perhaps the
most valuable of its features is that it provides a way
of keeping and transmitting both the audio signals and
their spatial characteristics in such a way that the
composer can think of spatial features independently
of the number of channels that will be available on a
particular performing situation. Another interesting
feature of Ambisonics is the set of operations of field
manipulation (which includes rotation, tilt, tumble,
dominance and mirroring) that can be performed on
its standard formats with relatively simple equations.
my_grainer~ may produce mono, stereo (intensity
panning), quad, nine channel or sixteen channel
outputs. The quad, nine channel and sixteen channel
outputs consist respectively of the four (W, X, Y, Z) or
the nine (W, X, Y, Z, R, S, T, U, V) or the sixteen (W,
X, Y, Z, R, S, T, U, V, K, L, M, N, L, O, P, Q) encoded
signals of an Ambisonic B first, second or third order
format, which must be further decoded by the user for
the available number of channels/loudspeakers of the
system using appropriate externals or abstractions. For
the Pure Data environment, an excellent choice would
be the HOA Library (see Hum [8]). An extra
(rightmost) signal outlet is always created to deliver a
reverberation send signal which can be used in
distance cues simulation (see below).
The user may spatialise a complete stream of grains
by feeding the input of ad hoc externals or
abstractions already available for that purpose using
the mono output of my_grainer~. Some externals
and/or abstractions that generate 3D trajectories based
on equations of specially relevant curves (such as
Lissajous, Spirals, Hypothrochoids, Epithrochoids,
etc.) may be used to drive the spatial location of a
stream of grains. The set of abstractions developed by
Pablo Cetta for Pure Data may be an excellent
choice for that purpose (see Cetta [4]).
If an individual spatial treatment of the grains is
desired, the Ambisonic output of my_grainer~
may be very effectively used, but it must be
noted that both Intensity Panning and
Ambisonic are angular location techniques. It is
also important to stress the fact that, as intensity
panning uses the IID cue (Interaural Intensity
Difference), it may be used to produce gradual
changes in the azimuth angle, but not in the
elevation angle of the spatial position of a
virtual sound source, while Ambisonics may be
used for both, azimuth and elevation location.
The distance simulation of my_grainer~ is
achieved by scaling the amplitude of each grain
by the inverse of a distance value that may also
have a random deviation. Each grain may also
be delayed by a duration computed taking into
account the virtual source distance and a sound
speed value provided by the user. Within closed
environments, the ratio between the direct and
reverberated signals and the time alignment
between the direct and the reverberated signals
are both simple, but perceptually effective
distance cues. These may be easily obtained by
feeding the input of a reverberation unit with an
output signal of my_grainer~ with neither of
the amplitude/time cues aforementioned
(which is always provided in the rightmost
outlet).
Curtis Roads ([13], pp. 221) deals with the
spatialisation of sound particles through two
approaches: one consisting of the simulation of
virtual spaces and sources, and the other
consisting of several particle modulations. In
what follows the latest will not be commented
because it exceeds the scope of this paper. The
former includes what Roads calls: “scattering
particles in virtual spaces” and “per grain
reverberation”. The capacities of my_grainer~
already described show clearly that it allows
both full 3D spatial movement of virtual
sources, as well as morphing of the dimensions
of sound clouds formed by small quanta of
spatially spread sound.
From the wide number of relations that may be
drawn between sound structure and sound
spatial quality, another interesting possibility
for which my_grainer~ was initially designed is
to work on the disruption of the spatial image of
virtual or acoustic sources, as described by
Gary Kendall (2009 [9]). Essentially, Kendall
states that the spatial treatment of sound in
electro acoustic music makes possible an
interplay with -among other features- the
perceptual grouping of the sonic events. This may be
achieved through the gradual modification of a GS
stream in order to produce the sensation of listening
either to a single sound source with a variable spatial
width, a spatially granulated texture or a group of
individual sound sources with or without coherent
sound images. In summary, the use of the GS
Techniques (time-domain decomposition and
recomposition) combined with the spatialisation
techniques may allow the production of sound images
which have not only interesting timbral qualities but
also surprising and even paradoxical spatial features.
4 Conclusions
my_grainer~ is an external that extends the
possibilities of GS offering features that are not
present in other existing abstractions/externals in the
Pure Data programming environment, specially in the
spatial treatment of sound. Since the external is in
continuous development, its code may be further
improved in order to speed up its performance. The
external was tested in several performance situations
showing robustness and efficiency and will allow
composers to explore and experiment with spatial GS
extensively. Some features of the external were also
designed taking into account its possible use on
concatenative synthesis, also combined with
spatialisation, which may be one of the future lines of
development.
5 Acknowledgements
Our thanks go to the Universidad Nacional de
Quilmes, Buenos Aires, Argentina for supporting the
Research Program in which this development is being
accomplished.
References
[1] J. Batty, K. Horn and S. Greuter : Audiovisual
granular synthesis: micro relationships between
sound and image. Proceedings of The 9th
Australasian Conference on Interactive
Entertainment: Matters of Life and Death, (p. 8),
ACM, Australia, 2013.
[2] Oeyvind Brandtsegg: Applications using the
partikkel opcode.
http://oeyvind.teks.no/results/applications/partikkela
pplications.htm (Last access: 09/2016)
[3] Ivica Ico Bukvic et al: disis_munger~ (a.k.a.
munger1~) source code, binaries and
documentation.
https://github.com/gilbertohasnofb/pd-abstractions-
and-libraries/tree/master/disis_munger~ (Last
access: 09/2016)
[4] Pablo Cetta: “Trayectorias espaciales a
partir de curvas notables”. Ideas Sónicas,
CMMAS (Centro Mexicano para la Música y
las Artes Sonoras), Morelia, México.
http://sonicideas.org/info.php?vol=8&num=
16&secc=notes (Last access: 09/2016)
[5] Alberto Del Campo: “Microsound”, In Scott
Wilson, David Cottle, and Nick Collins: The
SuperCollider Book, The MIT Press, London,
UK. pp. 463-504, 2011.
[6] M. A. Gerzon: “Periphony: With-height
Sound Reproduction”, Journal of the Audio
Engineering Society, vol. 21, N° 1, pp. 2-10,
1973.
[7] Joaquim Heintz et al: Csound FLOSS
Manual.
http://write.flossmanuals.net/csound/f-granul
ar-synthesis/ (Last access: 09/2016)
[8] Dionys Hum: HOA Library.
http://www.mshparisnord.fr/hoalibrary/en/do
wnloads/puredata/ (Last access: 09/2016)
[9] Gary Kendall: "La interpretación de la
espacialización electroacústica: atributos
espaciales y esquemas auditivos”. Gustavo
Basso, Oscar Pablo Di Liscia y Juan Pampin
(Eds.). Música y espacio: ciencia, tecnología
y estética. Bernal: Editorial de la Universidad
Nacional de Quilmes, pp.241-259, 2009.
[10] Dave Malham: “El espacio acústico
tridimensional y su simulación por medio de
Ambisonics”. Gustavo Basso, Oscar Pablo
Di Liscia y Juan Pampin (Eds.). Música y
espacio: ciencia, tecnología y estética.
Bernal: Editorial de la Universidad Nacional
de Quilmes. pp 161-201, 2009.
[11] Nick Mariette: Some Granular Synthesis
Implementations In Pd.
https://puredata.info/Members/nmariette/gran
ular-implementations/?searchterm=Syncgrain
~%20Synchronous%20Granular%20Synthes
is (Last access: 09/2016)
[12] Juan Pampin: A line.
https://dxarts.washington.edu/creative-work/
line (Last access: 09/2016)
[13]C. Roads: Microsound, The MIT Press,
England, 2004.
[14] Diemo Schwarz: "Current research in
concatenative sound synthesis". Proceedings
of the International Computer Music
Conference (ICMC), Barcelona, Spain,
September 5-9, 2005.
[15] SuperCollider 3.8dev Documentation: [17] Xenakios: HourGlass granulator
http://doc.sccode.org/Classes/GrainBuf.html download
http://doc.sccode.org/Classes/GrainFM.html page.https://xenakios.wordpress.com/2012/0
http://doc.sccode.org/Classes/GrainIn.html 4/29/hourglass-1-0-1-released/ (Last access:
http://doc.sccode.org/Classes/GrainSin.html 09/2016)
(Last access: 09/2016)
[16] Barry Vercoe et al: The Canonnical Csound
Reference Manual.
http://www.csounds.com/manual/html/SiggenGranu
lar.html (Last access: 09/2016)