Professional Documents
Culture Documents
Xiao Xiao
Abstract
This thesis introduces and examines methods for the capture and
reproduction of music on the piano that maintain a tight coupling
between the body, the sound, and the physical instrument. For expert
musicians, the body plays an indispensable role in the physical act of
playing and the understanding of both musical structure and expressivity.
However, many music learning technologies mistakenly assume that
playing music is “playing the notes” and neglect the role of the body in
the development of the musical mind.
Drawing from research in telepresence, tangible interfaces, and augmented
reality, I propose to bring the body back into the picture, literally and
metaphorically, by augmenting a digital player piano with projection-
mapping. My platform synchronizes dynamic imagery with the piano’s
moving keys and acoustic sounds.
I here focus on two main projects: MirrorFugue and Andante. Inspired by
reflections on the lacquered surfaces of a grand piano, MirrorFugue
simulates the presence of a virtual pianist whose reflection is actually
playing the physically moving keys. It encourages anyone to take the seat
left empty at the piano, to feel in his or her own body how music is
expressed through the body of the performer, and to play along. Andante
presents music as miniature figures that appear to walk and dance on the
piano keyboard, physically striking a key with each stop. It conveys the
expressivity in rhythms and phrases as well as musical structures through
the bodies and movements of the figures. Both installations are designed
as immersive “sandboxes” for the playful exploration of musical ideas.
Beyond my projects, this thesis explores the parallels between music
learning and learning and large. I discuss the connections between theories
of music learning (particularly Dalcroze Eurhythmics) with theories of
general mental development (Jean Piaget, Jerome Bruner, Seymour Papert,
and Marvin Minsky), as well as how strategies from music learning could
inform the art of learning in general.