Statistical Music Modeling Aimed at

Identification and Alignment

Riccardo Miotto, Nicola Montecchio, and Nicola Orio

Abstract. This paper describes a methodology for the statistical modeling of music
works. Starting from either the representation of the symbolic score or the audio
recording of a performance, a hidden Markov model is built to represent the corre-
sponding music work. The model can be used to identify unknown recordings and
to align them with the corresponding score. Experimental evaluation using a col-
lection of classical music recordings showed that this approach is effective in terms
of both identification and alignment. The methodology can be exploited as the core
component for a set of tools aimed at accessing and actively listening to a music
collection.

1 Introduction
The act of performing a music work, which has been coded in a music score by a
composer, can be considered as a process that converts score symbols into acoustic
features. To this aim, performers allow composers to communicate with the audi-
ence by transforming a sequence of symbols into something that can be perceived:
the sound. While playing the role of intermediaries, musicians can also add their
own interpretation to the music work, because music is both a composing and a
performing art. The degree of freedom allowed to the performers is mostly genre-
dependent. For instance, Western art music – also called tonal Western music, or

Riccardo Miotto
Department of Information Engineering, University of Padova
e-mail: miottori@dei.unipd.it
Nicola Montecchio
Department of Information Engineering, University of Padova
e-mail: montecc2@dei.unipd.it
Nicola Orio
Department of Information Engineering, University of Padova
e-mail: orio@dei.unipd.it

Z.W. Raś and A.A. Wieczorkowska (Eds.): Adv. in Music Inform. Retrieval, SCI 274, pp. 187–212.
springerlink.com c Springer-Verlag Berlin Heidelberg 2010
188 R. Miotto, N. Montecchio, and N. Orio

more generally classical music – imposes a strict adherence to the score, which gen-
erally prescribes all the notes that have to be played by each instrument (in most
cases it also indicates which are the instruments associated to each part) and gives
indications about timing, articulation, and dynamics. Other genres, such as jazz or
fusion, grant performers the freedom to make substantial changes to the main mu-
sic dimensions – melody, rhythm, and even harmony – and music scores usually do
not even represent articulation or dynamics. In between these two extremes, gen-
res such as pop and rock let performers change the arrangement and the orchestra-
tion, but usually the main melody and the chord progression are subject to minor
modifications.
While listening, the audience can associate a performance to a given composition
according to different strategies, which depend on the degree of personal interpreta-
tion that is expected by the performers for a given music genre. Similar strategies are
exploited when the listeners try to follow an ongoing performance along a symbolic
score. In both cases the expected acoustic parameters, that are inferred from the sym-
bolic representation of the music work, are compared with the perceived acoustic
parameters, and a number of hypotheses are formulated considering the probability
that differences are the result of the personal interpretation by the performers. This
process of identification and alignment is probably related to the central role that
symbolic representation plays in Western art music. Although a music score is only
an approximate representation of a music work, because it cannot express all the
possible nuances of a music performance [21], the score is often considered as the
ideal version of a music work, to which performances are only approximations [16].
Given these considerations, it can be assumed that there is a statistical depen-
dence between the symbolic representation and the acoustic performance of a given
music work. The degree of correlation between symbols and acoustic parameters is
clearly connected to the freedom of interpretation granted to performers. Moreover,
it can be assumed that a statistical dependence exists between the acoustic parame-
ters of two different performances of the same music work.
This paper presents an approach to statistical music modeling based on an ap-
plication of Hidden Markov Models (HMMs). To this end, we provide a unified
methodology that allows us to generate a HMM, which is the abstraction of a music
work and models the possible differences of its performances, starting from either
a symbolic representation of the score or an acoustic recording. Once a HMM is
created to represent a music score, it can be used to simulate the listener’s behavior
both in identifying a music work given a performance and in following a perfor-
mance along the corresponding score.
As an initial step towards the definition of the statistical dependence between
different representations of a music work, we focus on Western art music that, as
previously mentioned, has a clear definition of the musical parameters that can or
cannot be modified by performers. This genre is also particularly suitable for the
main application domain that we envisage, which is the access to music cultural
heritage in an educational context.
This paper is structured as follows. In Sect. 2 we provide a short review of
the main problems addressed through HMMs and the description of different