Audio Engineering Society

Convention e-Brief 193

Presented at the 138th Convention
2015 May 7–10 Warsaw, Poland

This Engineering Brief was selected on the basis of a submitted synopsis. The author is solely responsible for its presentation,
and the AES takes no responsibility for the contents. All rights reserved. Reproduction of this paper, or any portion thereof, is not
permitted without direct permission from the Audio Engineering Society.

  Immersive Sound Design using Particle

Systems
1
Nuno Fonseca
1
ESTG/CIIC, Polytechnic Institute of Leiria, Portugal
nuno.fonseca@ipleiria.pt

ABSTRACT

With the release of major immersive audio formats for cinema, including Auro-3D and Dolby Atmos, a new sound
dimension is getting the attention of sound professionals - height. Although traditional panning techniques are still
possible, more interesting approaches could be used to better explore space.

Particles systems, which are widely used on computer graphics, could present themselves as a very interesting sound
design approach. With immersive virtual microphones, capable of supporting many different setups, perfect sound
and space coherence can be obtained.

By controlling the particle system, instead of individual sound sources, a high number of sounds can be rendered in
3D. A particle system software was created, capable of running highly complex situations with up to several
millions of sound sources, which is currently under testing by major Hollywood studios.

and it is the particle system that creates and controls

1. PARTICLE SYSTEMS
Particle Systems are a common tool in visual effects and
computer graphics to recreate fuzzy/shapeless objects
like smoke, rain, fire or dust. Since most of these
objects are actually created by a high number of smaller
particles (e.g. rain drop, dust grain, smoke grain), it is
not practical to manually control each particle
individually. To handle this problem, particle systems
were invented by William Reeves [1], a researcher at
LucasFilms. Instead of controlling individual objects,
the user controls some parameters of a particle system,
each individual particle.
The same concept can be used for audio applications
[2], where each particle represents a sound source,
instead of representing a visual object. Also, one or
more virtual microphones must exist to capture the
sound of the particles, replacing the virtual cameras.
The sound “captured” by each virtual microphone must
take into consideration the microphone direction, the
delay and attenuation created by distance, and any
Doppler-related effect.
Fonseca
2. PARTICLE SYSTEMS AS A SOUND
DESIGN TOOL
A software was developed that is able to implement
particle systems for audio applications (Figure 1),
allowing the use of this technique for sound design
tasks.
Figure 1 - Particle system software for audio
applications
For each group of particles, the user may specify:
• The total number of particles
• The original audio files, which are randomly
assigned to the particles (each particle
randomly selects one of the available audio
files for sound reproduction)
• The initial position of the particles, ranging
from a single point in space to a particle
distribution inside 2D areas or 3D volumes
(rectangle, box, circle line, sphere, sphere
surface, torus, etc.)
• Movement modifiers, which apply movement
to the particles (including random velocities
and directions)
• Audio modifiers, which apply random changes
to each particle to create sound differences
between them (random gain, delay, EQ, sub-
band, grain, time/pitch).
3. IMMERSIVE SOUND
Immersive sound is the designation that was given to
the existence of sound in a 3 dimensional world,
including a height dimension. Although immersive
audio systems are not new, the release of commercial
immersive formats for cinema like Auro-3D and Dolby
Atmos, had bring a completely new interest on the
AES 138th Convention, Warsaw, Poland, 2015 May 7–10
Page 2 of 3
Immersive Sound Design using Particle Systems
subject, allowing filmmakers to include an immersive
sound experience on their movies.
There are 2 main approaches regarding immersive
audio: channel-based audio and object-based audio.
Channel-based audio uses the traditional channel
approach, where all sound sources are mixed to output
channels, where each output channel corresponds to a
speaker (or an array of speakers) that will exist at a
predefined position. Object-based audio uses a different
approach, using metadata. Instead of using channels
with predefined positions, each audio object will have
the desired audio stream and metadata with time varying
information on how that audio stream should be played
(position in space, apparent size, etc.).
3.1. Channel-based Audio
Immersive channel-based audio is supported using
virtual microphones with Vector Base Amplitude
Panning (VBAP) [3]. Depending on the desired
immersive speaker setup (5.1, 7.1, 9.1, 11.1, 22.2, etc.),
the sound of each particle is “captured” within the
triangle of speakers that best “enclosure” that particle
direction. Since many immersive speaker setups don’t
include speakers bellow the z=0 horizontal plane, to
better address situations where sound is reproduced by
particles with z<0, a virtual speaker is considered,
which will later remap its signal to existing z=0
speakers, with a gain that depends on coverage angle of
each speaker.
3.2. Object-based Audio
In most situations, the number of particles is much
higher than the maximum number of supported audio
objects of a system (for instance, Dolby Atmos only
support up to 128 audio inputs [4]). To overcome this
limitation, a hybrid channel/object approach is used: a
high number of channels are used (e.g. 31.1) and each
channel is used as an audio object, but with a static
position, allowing a better space resolution than
traditional channel-based formats.
4. DESIGNING SOUND
Particle systems can be used in many different contexts
within sound design. Two specific types of applications
are presented that clearly shows the advantages of such
approach, but many other applications exist, ranging
from simple Doppler effects to complex 3D sound
environments.
Fonseca Immersive Sound Design using Particle Systems

4.1. Massive Sound Layering 6. REFERENCES

In some situations, the sound designer may want to
layer a high number of individual sounds as a way to
create ambient sounds to represent some environments
(e.g. a medieval battle, a restaurant, etc.). Instead of
layering sounds manually, creating some tens of audio
tracks, particle systems could be used, spreading up to
thousands or millions of particles over a virtual space.
For instance, to recreate the sound of a medieval
battlefield, tens of thousands of particles could be
spread over a square mile, using tens or hundreds of
existing sounds files (swords, arrows, horses, screams,
impacts, etc.). By spreading particles above the
horizontal plane (e.g. soldiers on a castle) or moving
particles above the virtual microphone (ex: arrows
passing by), true immersive sound can be obtained.
4.2. Granular Synthesis
Some sounds are generated by a high number of similar
sound events. For instance, the sound of a crowd
applauding is made of a high number of hands claps.
The same way that particle systems are use in computer
graphics to recreate many grains (dust, smoke, etc.), the
same idea may be used in audio, using small audio
grains as particles.
[1] W.T.Reeves, “Particle Systems—a Technique for
Modeling a Class of Fuzzy Objects”, ACM
Transactions on Graphics (TOG), Volume 2 Issue
2, pp. 91-108, April 1983.
[2] Nuno Fonseca, "3D Particle Systems for audio
Applications", DAFx 2013 - 16th International
Conference on Digital Audio Effects, Ireland, 2013.
[3] Ville Pulkki, "Virtual Sound Source Positioning
Using Vector Base Amplitude Panning," J. Audio
Eng. Soc., vol. 45, pp. 456-466, (1997 June)
[4] Dolby, “White Paper - Dolby® Atmos® Next-
Generation Audio for Cinema,” Issue 3, 2014,
http://www.dolby.com/uploadedFiles/Assets/US/D
oc/Professional/Dolby-Atmos-Next-Generation-
Audio-for-Cinema.pdf (as seen in May 2014).

By using one or more audio files with the source

material (e.g. hand claps), a high number of particles
can be created per second, adding several audio
modifiers to obtain high diversity (random gains,
random delays, random EQ, random pitch-shifting). A
special granular synthesis audio modifier can be used,
allowing each particle to reproduce only a small time
fragment of the original audio file (instead of
reproducing the entire audio file).

5. CONCLUSION

Instead of designing sound originally in mono/stereo

and using mixing techniques to try to add more “space
information”, particle systems allows a new creative
mindset, offering a true coherence between sound and
space. And by adding randomness to the behavior of the
sound sources, better productivity (when compared with
manual approaches) and more organic results are
obtained.

Currently, the software implementation of a particle

system created by the author is under tests in several on
major Hollywood studios.

AES 138th Convention, Warsaw, Poland, 2015 May 7–10

Page 3 of 3