Amplifying Reflective Thinking in Musical Performance

Andrew Johnston, Shigeki Amitani and Ernest Edmonds

Creativity & Cognition Studios, Faculty of IT, University of Technology, SydneyPO Box 123 Broadway, NSW 2007, AUSTRALIA{aj, shigeki}@it.uts.edu.au, ernest@ernestedmonds.com

ABSTRACT

In this paper we report on the development of tools thatencourage both a creative and reflective approach to music-making and musical skill development. A theoreticalapproach to musical skill development is outlined and previous work in the area of music visualisation isdiscussed. In addition the characterisation of music performance as a type of design problem is discussed andthe implications of this position for the design of tools for musicians are outlined. Prototype tools, the design of whichis informed by the theories and previous work, aredescribed and some preliminary evaluation of their effectiveness is discussed. Future directions are outlined.

Categories & Subject Descriptors:

H5.5. Sound andMusic Computing.

General Terms:

Design; Performance;

Keywords:

Reflection-in/on-action; musical performance;design problem; visualisation;

INTRODUCTION

The authors are going to explore the possibilities of computers through implementations of systems for supporting musical performance. Our hypothesis is thatsystems that provide musicians with visual feedbacks promote reflective thinking on their performance. The principal research question is how computers play the roleof a catalyst, or a stimulant [7], in order to amplifyreflective thinking in musical performance.

A THEORETICAL BASIS FOR INSTRUMENTAL MUSICTOOLS

In order that tools for instrumental musicians are genuinelyuseful, it is important for software designers to consider carefully the theoretical position upon which their tools are based. An approach which fails to explicitly consider the pedagogical implications of tool use may lead to animplicit, simplistic theory of music learning beingembedded into the system, leading to unsatisfactoryoutcomes for all concerned.Because of rapid technical advances it is understandablethat there can be a tendency for tools to be developed basedon technical feasibility rather than genuine utility and for this reason we feel it is necessary to outline the conceptualframework underpinning our work. In this section we firstoutline pedagogical approaches to musical skilldevelopment and then outline techniques for encouraging areflective approach to creative work. Previous work in thearea of using visualisation techniques to facilitate different perspectives on complex or ambiguous data is discussed asare some key attempts at creating useful musicalvisualisations.

Pedagogical Approaches to Instrumental Music

Instrumental musicians require skills in two key areas.Firstly they must develop the ability to physicallymanipulate their body and instrument to produce musicalsounds and secondly they require creative skills in order that the music they produce is interesting to others. Whilevarious styles of music may emphasise either the craft of instrumental music or the artistic side of musical skill togreater or lesser degrees, musicians generally require a highlevel of ability in both areas.Broadly speaking, pedagogical approaches to thedevelopment of the necessary physical skills to play aninstrument can be divided into two camps. The firstemphasises the need to understand the physiology of instrumental technique in order to facilitate consciouscontrol of the various muscles involved so that “correct”technique can be used. Perhaps due to a desire to approachinstrumental music in a more “scientific” way, there is oftena tendency for musicians to attempt to take a reductionistapproach to improving their playing. For example, singersmay attempt to support their sounds by attempting toconsciously control their diaphragm in some way in order to improve their range, volume or tone.Kohut describes this approach as the “physiological-analysis-conscious-control” method [18]; in whichmusicians attempt to understand the physical actionsinvolved in music making in detail and exert control over them consciously while playing. The key assumptions hereare that



Complex involuntary muscle manoeuvres can becontrolled consciously;



Conscious control will lead to improved performance;



Conscious control is best achieved by attempting tocontrol the individual muscles/organs involved;



A detailed intellectual understanding of the physicalactions involved in playing is the key to improving performance.The second approach, dubbed the “imitation method” [18],rejects these assumptions, arguing that



The muscle manoeuvres involved are too complexand subtle to be meaningfully controlled by theconscious mind and that many muscles (such as thediaphragm) cannot be controlled directly anyway.Recent studies of the physiology of memory seem toindicate that different regions of the brain areresponsible for what is termed

implicit

and

explicit

memory and that motor skills are a form of the non-conscious implicit memory. Experiments withamnesiac patients for example show that patients whohave no conscious recollection of learning a skill (i.e.they forget the training sessions during which the skillwas learned) still demonstrate normal motor-skilllearning ability [28];



Attempting to consciously control the minutiae of physical actions that take place while playing amusical instrument is at best likely to lead to a tense,mechanical-sounding musical outcome and at worstto “paralysis by analysis” [10, 18, 30], where themusician becomes overwhelmed by the complexity of detailed muscle control and loses the ability to playeven simple tunes on their instrument;



Improved performance results from setting andrefining specific musical goals rather thanconsciously attempting to control physical actions at alow level;



A detailed intellectual understanding of the physicalactions involved in playing is not necessary.For these reasons the trend in music pedagogy has beentowards an approach based on the “natural learning process” (NLP) [12] which emphasises the importance of leaving the complexities of muscle control to thesubconscious so that the conscious mind remains free to sethigh-level musical goals. The technique for developingnew skills is based on imitation, with a strong emphasis onmental musical goal setting and excellence of role models.Teachers taking this approach for example would tend tospend more time on playing for students during lessons andencouraging them to try to copy aspects of the teacher’ssound, instrumental technique or musical phrasing andwould discourage discussion of physical aspects of instrumental technique.Of course, the use of imitation to develop physical andmusical skills does not preclude development of anindividual style. An individual musician’s mental image of musical ideals for given situations develops as they learn of different approaches through listening and watching performances by musicians on their instrument and others.Thus the musician may in effect choose a sound for a particular section of music which has characteristics of several other musicians but which is nevertheless unique.That is to say, the musical approach is influenced by manyrole models but retains qualities unique to the individualmusician.In this approach then, the musician gradually develops physical skills and an individual style as they build up amental library of “target sounds” which trigger theappropriate physical responses. This mental library isreflected upon and refined through experience and exposureto new ideas, both musical and extra-musical. Thusregardless of the style of music performed, creativity is afundamental part of musical skill development, asmusicians constantly work towards an ideal sound whichitself is being constantly refined. Even when musicians are playing music composed by someone else, creativity isrequired in the interpretation of the music notation chosen by the composer. The subtleties of musical phrasing -including tempo, articulation and dynamics – cannot beadequately represented by musical notation and thus theinterpreting musician has a significant influence on the finalmusical result [11]. It could be said that in effect the performance is collaboration between composer andmusician.

Information Visualisation Research

The research field of information visualisation has beendeveloped as computational power grows. Especially inHuman-Computer Interaction field, the design of visualisations and interactions has been one of the centralissues [5]. In the HCI research field, the aim of informationvisualisation is to facilitate people to understand largequantity of information and its tendencies.There have been several approaches to using thecapabilities of computers to visualise aspects of musical performances to help musicians develop their skills. Theapproaches range from what might be called the “scientific”approach, where audio sounds are analysed and displayedin the form of graphs to more “artistic” visualisations wherethe correlation between musical input and computer displayis more abstract.The “Sing-and-See” program, taking the former approach,[34] is designed for singers and singing teachers. The user sings into a microphone and the connected computer displays visualisations of various aspects of the sound. Theavailable displays are a piano keyboard and traditionalmusic notation staff which highlight the note currently being sung, a pitch-time graph showing the pitch of thesung note in comparison to the standard equal-temperament

keyboard and a spectrogram and real-time spectrum displayshowing power in the voice at different frequencies.Another program which produces similar output -VoceVista (http://www.vocevista.com) - has been used toanalyse various aspects of vocal performances [21].Successfully incorporating these tools into everyday practice requires careful thought, as the complicated natureof the spectrogram display is open to interpretation and mayencourage an overly analytical approach.A project which illustrates a more abstract approach to themapping between sound and computer response is the“Singing Tree” [23]. The singing tree provides both audioand visual feedback to singers in real time. The singer sings a pitch into a microphone and if the pitch is steady thecomputer provides an accompaniment of consonantharmonies from string, woodwind and vocal chorus. If the pitch deviates, the computer introduces gradually increasingdissonance to the accompaniment including brassier andmore percussive instruments and more chaotic rhythms. Inaddition to the audio feedback, when the singer’s note issteady the computer generates video which is designed togive the impression of moving forwards towards anidentifiable goal- an opening flower for example. If asteady pitch is not maintained the video reverses. Such anapproach emphasises a playful approach and the link between audio input and the audio/visual output is intendedto be less deterministic than the more “scientific” mapping between sound and vision used in tools such as “Sing andSee”.A prototype tool which analyses performances from another perspective has been created by Nishimoto and Oshima[22]. Their “Music-Aide” program takes standard MIDI(Musical Instrument Digital Interface) input and representsthe musical phrases produced by the musician graphically.Different components of the phrases, such as consonant or dissonant notes are displayed in such a way that the relation between these components can be seen.For example, it might be that when the improvisingmusician plays phrases containing predominantly firsts andfifths (i.e. ‘monochrome chord tones’), she never includesthe tritone. Music-Aide would show this characteristicgraphically. It is hoped that the musician will therefore beable to notice aspects of their improvising that they mightotherwise be unaware of, and thus help them to overcome bad habits or discover new ways of playing.Another way that Music-Aide might be able to help is infinding new directions and styles. For example, if amusician noticed that the representation of their improvisations always had an empty space in a particular part of the display, they could work out phrases to play thatwould move the representation into that space. In this way, perhaps tools such as Music-Aide could stimulate the performer to try a style of playing that they had previouslynot considered.A tool which displays a view of the relationship betweentempo and loudness rather than harmonic aspects isdescribed by Langner and Goebl [19]. This tool shows agraph with the x-axis representing tempo and the y-axis thedynamic level. As the music is played, a dot moves aroundwithin the graph. As the tempo and loudness vary, the dotmoves around the screen, leaving a kind of three-dimensional “trail” behind it that very effectively illustratesthe high-level shaping of phrases by the performer. For example, using such a tool the performer might identify patterns in their playing – always slowing down when playing at lower dynamic levels perhaps – that they did notdetect aurally.

Musical Performance as a Design Problem

Design problems are characterised with ill-definedness andopen-endedness, that is, one can neither specify a design problem completely before starting to solve it, nor articulateknowledge necessary for a design a priori. Designers haveto gradually refine both the problem specification and thedesign solution at the same time. Musical performance can be characterised as a design problem because no single"ideal" performance can be defined a priori. Musicianschange their performances along with the context [6], thehistory of interactions among the musicians, audiences and produced sound.Schoen [29] has explained a design process as "Seeing-Drawing-Seeing Cycle". Design process starts from"Seeing", proceeds to "Drawing", or generating the partialsolution to the design problem, returns to "Seeing" andrepeats this cycle. This process is regarded as a "dialogue" between the designer and his/her material. In musical performance, a performer listens to existing sound (thecontext), conceptualises and plays a phrase as a partialsolution to the musical “problem” posed by the context, andlistens to the result. The cyclic process is "listening- playing-listening". As a way of reducing the chance the performer will become distracted by self-talk and possiblytrigger “paralysis by analysis”, Jacobs [30] suggests the performer should picture in their mind what should happenand leave it to the subconscious to take care of the details.There are personal accounts of the musical flow of Mozart'scomposition [1]:

“My subject enlarges itself, becomes methodised and defined, and the whole, though it be long, stands almost complete and finished in my mind, so that I can survey it,like a fine picture or a beautiful statute, at a glance.”“Nor do I hear in my imagination the parts successively; I hear them all at once. What a delight this is! All thisinventing, this producing, takes place in a pleasing, livelydream.”

Though this anecdote may be a desirable goal for musicians, it is not the case in reality. Musicians reflect ontheir performance and adjust their performance in a proper way. This is borne out by the fact that concert halls are