Cornell University

CS 6650: Computational Motion Semester Project

Rigid Body Sound Synthesis
Authors:
Charles Moyes
Shentong Wang
Supervisor:
Dr. Doug L. James
April 26, 2011
Contents
1 Abstract 2
2 Math Library 3
2.1 Dense Solver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Armadillo Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 Sparse Matrix Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.4 SVDLIBC C Library . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 Sound Synthesis 5
3.1 Finite Element Model Matrix Formation . . . . . . . . . . . . . . . . . . . . 5
3.2 Material Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3 Modal Synthesis Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.4 Audio Playback Using SDL . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4 Rigid Body Dynamics 8
4.1 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2 Resting Contact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2.1 Analytical Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.2.2 Projected Relaxed Gauss-Seidel Solver . . . . . . . . . . . . . . . . . 8
4.3 Collision Detection and Response . . . . . . . . . . . . . . . . . . . . . . . . 9
4.3.1 Collision Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.3.2 Newtons Law of Restitution . . . . . . . . . . . . . . . . . . . . . . . 9
4.3.3 Tangential Coulomb Friction . . . . . . . . . . . . . . . . . . . . . . . 9
4.4 Open Dynamics Engine (ODE) Integration . . . . . . . . . . . . . . . . . . . 10
5 Graphics Engine 11
5.1 Test GUI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.1.1 Keyboard Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.2 OpenGL Renderer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.2.1 Keyboard Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
6 Future Plans 15
7 Conclusion 16
Bibliography 17
1
Chapter 1
Abstract
The basic idea of the project is to implement a rigid body sound synthesis engine in C++
using OpenGL and a rigid body dynamics simulation. Modal synthesis in the frequency-
domain will be used to generate rigid body contact sounds based on the contact forces and
positions applied to objects in the simulation. The Armadillo linear algebra library will be
used to perform routine matrix decompositions such as Cholesky factorizations and singular
value decompositions (SVDs). The SDL library will be used for audio output and graphical
rendering. Another goal is to build an interesting simulation to show o the applications of
the engine.
Figure 1.1: Proof-of-concept image from [12].
2
Chapter 2
Math Library
The project required linear algebra routines for forming the mass and stiness matrices,
along with solving the eigenvalue problem associated with modal analysis. For this project,
we implemented our own dense and sparse matrix classes, along with using the Armadillo
linear algebra library (found at http://arma.sourceforge.net/) and the SVDLIBC sparse
singular value decomposition library (found at http://tedlab.mit.edu/
~
dr/SVDLIBC/).
We have implemented both a sparse and a dense matrix solver. We found that the dense
solver works best with smaller tetrahedral meshes.
2.1 Dense Solver
The dense solver was implemented rst using the Armadillo library to have a metric for
testing the sparse matrix routines. Meanwhile, the sparse solver was developed concurrently.
2.2 Armadillo Integration
Armadillo was used for the dense solver for computing the Cholesky decomposition of the
mass matrix. For the sparse matrix solver, we generate a diagonal lumped mass ma-
trix, avoiding the need to perform an explicit Cholesky decomposition. In this case, the
Cholesky decomposition is trivially the square root of the diagonal values of the matrix.
The Eigen linear algebra library (http://eigen.tuxfamily.org/) was also investigated
for this project, but we found that the Armadillo API was easier to use with our code.
2.3 Sparse Matrix Library
Our sparse matrix library initially handled both static and dynamic matrices. We used a
storage format very similar to the Harwell-Boeing format. Column indices separate an array
of non-zero values into partitions, where each group resides in a dierent column. Within
each column, each non-zero value in the sparse matrix has a row index associated with it.
Very thorough test batteries were written to ensure the correctness of this code in order to
prevent tedious debugging further down the road.
2.4 SVDLIBC C Library
At rst, we tried developing our own sparse matrix eigenvalue decomposition code, but
after consulting with Charles Van Loan and David Bindel, we decided that this endeavor
would be a separate project in itself. The SVDLIBC library was used for performing
sparse eigenvalue decompositions. It uses the Lanczos algorithm for computing the singular
3
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
value decomposition, and it handles both dense and sparse matrices. Code was written
to generate sparse matrices in the Harwell-Boeing binary format for use with this library.
Several Fortran libraries were investigated, including PROPACK, TRLan, and LAPACK.
We found that SVDLIBC was the easiest to use, so we went with it instead.
4
Chapter 3
Sound Synthesis
3.1 Finite Element Model Matrix Formation
We used [11] as a reference for constructing the matrices for tetrahedral nite element mod-
els. We discretize our mesh into tetrahedron in order to perform modal analysis. The
mesh vertices and tetrahedron indices were loaded directly from binary les generated using
Changxi Zhengs IsoStuffer utility [15]. We used linear basis functions as described in
the paper. We wrote code to assemble the mass and stiness matrices as suggested. The
element mass matrix M was computed as:
m
[ij]ab
=

2

p
[i]a
p
[j]b
=
vol
20
(1 +
ij
)
ab
(3.1)
and the lumped mass matrix was formed as follows:
m
lumped
[ij]ab
=
vol
4

ij

ab
(3.2)
The stiness matrix K was formed using:
k
[ij]ab
=
f
[i]a
p
[j]b

p=p
rest
=
vol
2
(
ia

jb
+
ib

ja
+
3

k=1

ik

jk

ab
) (3.3)
where represents the Barycentric coordinates within a tetrahedral element (dening a
linear shape function for that element), and the volume of a tetrahedral was computed using
the triple scalar product:
vol =
1
6
[(m
[2]
m
[1]
) (m
[3]
m
[1]
)] (m
[4]
m
[1]
) (3.4)
where m
[
i] is the position in material (or object) coordinates of element node i.
The damping matrix C is dened as a linear combination of the stiness and mass
matrices (so-called Rayleigh damping):
C =
1
K+
2
M (3.5)
3.2 Material Parameters
The so-called Lame parameters and dene the properties of the material in its elastic
stress tensor (assuming the material is isotropic). Our simulation code allows one to tune
these parameters, along with the material density and its Rayleigh damping coecients.
Example material parameters for various objects can be found in the following table:
5
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Material (Pa) (Pa)
1

2
(Kg/m
3
)
Ceramic 3.99 10
9
2.05 10
9
1 10
6
10 2700
Plastic 2.49 10
10
1.28 10
10
1 10
6
50 2700
Aluminum 4.98 10
10
2.56 10
10
1 10
7
0 2700
Wood 5.00 10
8
1.00 10
8
8 10
6
50 750
Metal 4.99 10
10
2.56 10
10
1 10
7
0 2700
Table 3.1: Example Material Parameters
3.3 Modal Synthesis Calculations
Modal synthesis is performed by computing the oscillation modes of the nite element mesh.
A brief overview of the modal analysis calculations involved follows. Starting with a non-
linear mass/spring/damping force system:
K(d) +C(d,

d) +M(

d) = f (3.6)
The system is linearized (with a loss in accuracy when modelling thin shells without
modal coupling) into the following form:
Kd +C

d +M

d = f (3.7)
By letting M = LL
T
(Cholesky factorization) and L
1
KL
T
= VV
T
(Singular Value
Decomposition):
z + (
1
+
2
I) z + z = g (3.8)
where g = V
T
L
1
f .
The Rayleigh damping assumption allows the system to be diagonalized into a set of
second-order ordinary dierential equations:

i
z
i
+ (
1

i
+
2
) z
i
+ z
i
= g
i
(3.9)
The analytical solution for each mode was computed to determine its frequency (|Im(
i
)|)
and decay rate (Re(
i
)):
z
i
= c
1
e
t
+
i
+ c
2
e
t

i
=
(
1

i
+
2
)
_
(
1

i
+
2
)
2
4
i
2
(3.10)
Finally, given the c constants for mode reponses to projected impulses:
c
1
=
2tg
i

+
i

i
(3.11)
c
2
=
2tg
i

i

+
i
(3.12)
These c constants can be used to nd a time-varying oscillator equation for the audio wave-
form:
z
i
=
2tg
i
|Im(
i
)|
e
tRe(
i
) sin(t|Im(
i
)|)
(3.13)
6
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Thus, an eigenvalue decomposition can be used to nd the modal frequency and decay
rates of the mass-spring-damper system. Contact forces from the rigid body simulation can
be projected onto the vibration modes (the columns of L
T
V). As an optimization, only
the modes with frequencies within the audible frequency range of the human ear (20 Hz to
20 kHz) are used for sound synthesis computations.
3.4 Audio Playback Using SDL
The SDL audio library was used for cross-platform audio playback. The summed contribution
of each modes oscillator equation is used to ll up the audio buer, which is invoked using
a mix audio() callback function. A queue of modes contributing to the resultant waveform
are stored in the STL vector, modeQueue. A mutex lock sound lock protects this queue from
race conditions. Moreover, a sampling rate of 44,100 Hz was used, and the audio data is
stored internally using oating-point numbers until the nal mixdown where it is discretized
into 16-bit signed integers. Special care was taken to avoid clipping by scaling the resultant
waveform so that it lies within the dynamic range of the audio buer. This approach has
the weakness that an unappropriately chosen scaling value will still allow clipping to occur,
trading o dynamic range for safety against clipping. A smarter approach could be to
dynamically scale the resultant waveform (like audio compressors in sound engineering).
This resultant waveform represents the sound you hear.
7
Chapter 4
Rigid Body Dynamics
A rigid body dynamics simulation was implemented mostly using the results from [6]. The
rigid body dynamics simulation component of this project provided support for Newtonian
physics simulation. The physics engine uses a projected, relaxed Gauss-Seidel solver to
analytically determine contact forces:

b = J(u
t
+ tM
1

f
ext
) (4.1)
A = JM
1
J
T
(4.2)

= ncp(A,

b) (4.3)
u
t+1
= u
t
+M
1
J
T

+ tM
1

f
ext
(4.4)
s
t+1
= s
t
+ tSu
t+1
(4.5)
4.1 Numerical Integration
The Euler method of numerical integration is used to timestep the simulation by computing
updated velocities and positions for each body. It is represented in matrix-form as discussed
in the Erleben paper. We needed a stable integrator to solve the dynamics equations. Prof.
James suggested using a reverse Euler method. This worked ne for our needs. The Time-
Corrected Verlet method was also considered for use, along with the fourth-order Runge-
Kutta method.
x
n+1
= x
n
+ f(t
n
, v
n
)t (4.6)
v
n+1
= v
n
+ g(t
n
, x
n
)t (4.7)
4.2 Resting Contact
4.2.1 Analytical Method
Contact forces were computed analytically using a relaxed Gauss-Seidel solver. To improve
simulation stability, I also manually resolved penetration at each time step.
4.2.2 Projected Relaxed Gauss-Seidel Solver
The projected Gauss-Seidel solver was implemented using the optimizations suggested in the
Erleben paper.

k+1
i
=

i1
j=0
L
i,j

k+1
j

n1
j=i+1
U
i,j

k
j

b
i
D
i,i
(4.8)
8
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
The projection step enforces the friction cone and the linear complementarity problem
constraints:

k+1
i
= min(max(

lo
i
,

k+1
i
,

hi
i
) (4.9)
We added a relaxation term = 0.25 to the iteration step given in the paper.
4.3 Collision Detection and Response
4.3.1 Collision Detection
Initially, the FreeSOLID library (found at http://www.win.tue.nl/
~
gino/solid/) was
used for collision detection. However, the Gauss-Seidel solver often exhibited divergence,
necessitating the use of Velocity Shock Propagation (VSP) as described in [6]. Unfortunately,
the implementation of this algorithm requires knowledge of the penetration depth. For simple
cases like ground-sphere and ground-box collisions, this could be determined without any pre-
determined knowledge from the FreeSOLIDlibrary, but in cases like stacking (where VSP is
especially needed to maintain stability and improve convergence), the FreeSOLID librarys
lack of a penetration depth result prevented the development of a VSP implementation.
Unfortunately, this left us with either the option of implementing our own collision detection
library or using the Open Dynamics Engine (ODE) for our rigid body calculations. We opted
for the latter option.
4.3.2 Newtons Law of Restitution
Newtons Law of Restitution is used to compute the collision response impulses:
j =
(1 + e)v
AB
1
n
n n
_
1
M
A
+
1
M
B
_
+
__
I
1
A
(r
AP
n)
_
r
AP
+
_
I
1
B
(r
BP
n)
_
r
BP

n
(4.10)
It is represented in Matrix form as part of the

b vector used in the rigid body system
of equations discussed in the Erleben paper. This is an elegant way to add bouncing as a
collision response to the simulation.
4.3.3 Tangential Coulomb Friction
Coulomb friction was also incorporate in the rigid body solver using extra constraints on
enforced during the projection step as discussed in the Erleben paper:
F
f
F
n
(4.11)
9
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
4.4 Open Dynamics Engine (ODE) Integration
The Open Dynamics Engine was integrated using a wrapper for rigid objects (with tetra-
hedral meshes for sound generation). The contact forces, normals, and positions are rst
accumulated in the nearCallback() function. This function creates contact joints as spec-
ied in the ODE documentation. Then in the updatePhysicsState() function, the world
is time stepped, and the joint feedback functionality of ODE is used to retrieve the forces
stored in the callback function. The relative velocity and acceleration for the contacting
bodies are computed, and if the relative velocity is non-zero, then the bodies are not in
resting contact. This means that they are bouncing, and so a rigid body contact sound
should be triggered. To prevent artifacts from contact sounds that are triggered too quickly
in succession, a grace time of MIN SOUND REPEAT TIME (set to 0.25 seconds by default) is
used.
Following the procedure outlined in [12], the contact force is projected onto the modes by
transforming the world space contact position back to object space and nding the nearest
vertex in the tetrahedral mesh. This vertex is the excited node, and so the force is applied
to that node in the f vector. From there, the g vector is computed as outlined in section
3.3. The mode frequencies and rates are then determined, and if the frequency lies within
the audible hearing range, a new mode is added to the mode queue for the sound synthesis
code.
An interesting curiousity in the Open Dynamics Engine is that its integrator is only
stable with constant time steps. However, one can update the error reduction parameter
(ERP) and constraint force mixing (CFM) values as functions of the current time step size
t, as outlined in [2]:
ERP =
t k
t k + c
(4.12)
CFM =
1
t k + c
(4.13)
These updates allow ODEs numerical integrator to remain stable, even with variably-
sized time steps.
10
Chapter 5
Graphics Engine
Multiple user interfaces were developed for this project. Initially, a testing interface that
used SDLs 2D drawing capabilities was used to ensure that the modal synthesis code was
working properly. This test interface featured an on-screen oscilloscope and a visualization
of the excitation of each modal amplitude. Later, OpenGL 3D graphics and rigid body
dynamics were added to visualize the actual rigid body contact sounds being generated.
5.1 Test GUI
The test GUI has several features, including the features to generate test modes, produce an
A 440 Hz test tone, project test forces onto the tetrahedral mesh, and hear all of the modes
at once. Moreover, some of the parameters can be changed on-the-y without re-evaluating
the system matrices. A screenshot of the test GUI follows. The red bars represent the
relative amplitudes of the excited modes:
Figure 5.1: Sound Synthesis GUI
5.1.1 Keyboard Controls
A table outlining the basic keyboard controls of the test GUI follows:
5.2 OpenGL Renderer
The OpenGL renderer allows the user to spawn falling spheres and boxes to demo the rigid
body contact sound functionality of the sound synthesis engine. The 3D camera view can be
11
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Functionality Key
Exit [ESCAPE]
Play 20 Highest Frequency Modes [Q]
A 440 Hz Test Tone [W]
Clear Mode Queue [E]
Project Test Force [R]
Play All Modes At Once [T]
Increase Number of Modes [A]
Decrease Number of Modes [S]
Increase
1
[Z]
Decrease
1
[X]
Increase
2
[C]
Decrease
2
[V]
Increase Scale Factor [B]
Decrease Scale Factor [N]
Increase Lame Scale Factor [M]
Decrease Lame Scale Factor [,]
Increase Excited Node Index [.]
Decrease Excited Node Index [/]
Toggle Full-Screen [TAB]
Table 5.1: Test GUI Keyboard Controls
adjusted using keyboard and mouse controls. Moreover, the parameters can be tuned using
the keyboard controls, much like with the Test GUI. A screenshot of the OpenGL renderer
demonstrating a sample physics environment follows:
5.2.1 Keyboard Controls
A table outlining the basic keyboard controls of the OpenGL 3D renderer follows. Note that
the parameter tuning keyboard controls are identical to the Test GUI interface. Additionally,
by dragging with the left mouse button, the viewing direction can be manipulated.
12
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Figure 5.2: OpenGL 3D Graphics Engine
13
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
Functionality Key
Exit [ESCAPE]
Camera Forward [W]
Camera Backward [S]
Camera Left [A]
Camera Right [D]
Apply Test Force [SPACE]
Create Rigid Box [H]
Create Rigid Sphere [J]
Clear Mode Queue [Q]
Increase
1
[Z]
Decrease
1
[X]
Increase
2
[C]
Decrease
2
[V]
Increase Scale Factor [B]
Decrease Scale Factor [N]
Increase Lame Scale Factor [M]
Decrease Lame Scale Factor [,]
Increase Excited Node Index [.]
Decrease Excited Node Index [/]
Toggle Full-Screen [TAB]
Table 5.2: OpenGL 3D Renderer Keyboard Controls
14
Chapter 6
Future Plans
Several future plans are in place for expanding this project:
Investigate and implement a far-eld acoustic radiation model. [8]
Add support for non-linear mode coupling (as in harmonic shells). [4]
Parallelize the sound synthesis engine using GPGPU programming (CUDA or GLSL)
or multi-threading.
Investigate and implement fracture sound synthesis. [17]
Implement recent advances in modal synthesis such as contact damping and modal
re-distribution. [16]
15
Chapter 7
Conclusion
The project was successful in that rigid body contact sounds were able to be generated using
the model presented in [12]. A exible user interface was developed, and the results are that
objects within the virtual 3D environment are able to generate contact sounds accordingly.
One can envision applications for real-time rigid body sound synthesis in videogames and
animated movies. The audio in videogames is often limited to playing back a pre-recorded set
of sound samples when game events take place. The end result is that the same glass shat-
tering sample is played regardless of the fracture pattern or the actual in-game shattering
behavior. Furthermore, sound artists have to resort to manually recording and overdubbing
the audio tracks in lms. These processes could be automated if a suciently realistic model
for sound generation is used.
16
Bibliography
[1] David Bara. Fast contact force computation for nonpenetrating rigid bodies. In Pro-
ceedings of the 21st annual conference on Computer graphics and interactive techniques,
SIGGRAPH 94, pages 2334, New York, NY, USA, 1994. ACM.
[2] Barringer. Tweaking ode for variable time steps, 2007.
[3] Nicolas Bonneel, George Drettakis, Nicolas Tsingos, and Doug James. Fast modal
sounds with scalable frequency-domain synthesis, 2008.
[4] Jerey N. Chadwick, Steven S. An, and Doug L. James. Harmonic shells: a practical
nonlinear sound model for near-rigid thin shells. ACM Transactions on Graphics, 28,
2009.
[5] Kees Van Den Doel, Paul G. Kry, and Dinesh K. Pai. Foleyautomatic: Physically-based
sound eects for interactive simulation and animation. In in Computer Graphics (ACM
SIGGRAPH 01 Conference Proceedings, pages 537544. ACM Press, 2001.
[6] Kenny Erleben. Velocity-based shock propagation for multibody dynamics animation.
ACM Trans. Graph., 26, June 2007.
[7] Gene H. Golub and Charles F. Van Loan. Matrix Computations. The Johns Hopkins
University Press, 3rd edition, 1996.
[8] Doug L. James, Jernej Barbic, and Dinesh K. Pai. Precomputed acoustic trans-
fer: Output-sensitive, accurate sound generation for geometrically complex vibration
sources. ACM Transactions on Graphics (SIGGRAPH 2006), 25(3), August 2006.
[9] Danny M. Kaufman. Staggered projections for frictional contact in multibody systems,
2008.
[10] James F. OBrien, Perry R. Cook, and Georg Essl. Synthesizing sounds from physically
based motion. In Proceedings of ACM SIGGRAPH 2001, pages 529536, August 2001.
[11] James F. OBrien and Jessica K. Hodgins. Graphical modeling and animation of brittle
fracture. In Proceedings of the 26th annual conference on Computer graphics and in-
teractive techniques, SIGGRAPH 99, pages 137146, New York, NY, USA, 1999. ACM
Press/Addison-Wesley Publishing Co.
[12] James F. OBrien, Chen Shen, and Christine M. Gatchalian. Synthesizing sounds from
rigid-body simulations. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics
symposium on Computer animation, SCA 02, pages 175181, New York, NY, USA,
2002. ACM.
[13] Cecile Picard, Christian Frisson, Fran cois Faure, George Drettakis, and Paul G. Kry.
Advances in modal analysis using a robust and multiscale method. EURASIP J. Adv.
Signal Process, 2010:7:17:12, February 2010.
17
Charles Moyes (cwm55) and Shentong Wang (sw477) CS 6650: Semester Project
[14] David E. Stewart. Rigid-body dynamics with friction and impact. SIAM Rev., 42:339,
March 2000.
[15] Changxi Zheng. Tetrahedra generation.
[16] Changxi Zheng and Doug L. James. Toward high-quality modal contact sound, 2001.
[17] Changxi Zheng and Doug L. James. Rigid-body fracture sound with precomputed
soundbanks. ACM Trans. Graph., 29:69:169:13, July 2010.
18