Due to a family bereavement I was unable to attend Understanding VISUAL Music 2011 and present this paper - but

since it snapshots the next stage of my research Im making this draft version available - albeit that its been retrospectively tidied up a little.

The Augmented Tonoscope

Understanding VISUAL Music 2011 August 26th and 27th, 2011 Hexagram-Concordia Centre for Research Creation in Media Arts and Technologies in collaboration with the Department of Music Concordia University, Montreal, Canada
Author
Lewis Sykes Manchester Institute for Research and Innovation in Art and Design (MIRIAD) Faculty of Art & Design, Manchester Metropolitan University, Righton Building, Cavendish Street, Manchester, M15 6BG lewis@augmentedtonoscope.net

Abstract
The Augmented Tonoscope is an artistic study into the aesthetics of sound and vibration through its analog in visual form - the modal wave patterns of Cymatics. Key to the research is the design, fabrication and crafting of a sonically and visually responsive hybrid analogue/digital instrument that will produce dynamic Visual Music based on the physics, effects and manifestations of sound and vibration. This paper describes the first stage of the study - a series of artistic investigations into analogue tonoscope and digital tone generator design integrating light and camera control - driven by an artistic experimental method devised specifically for the project. The paper concludes with future directions for the research: Can the inherent geometries within sound provide a meaningful basis for Visual Music? Will augmenting these physical effects with virtual simulations realise a real-time correlation between the visual and the musical?

Introduction
Thanks so much for the invitation and the opportunity to present here - to be amongst peers who are familiar with the context of my research. Frankly its something of a relief that I dont have to preface my presentation with a definition of Visual Music while standing on one leg or try to fleetingly illustrate the bodies of work of artistic influences such as Oskar Fischinger or John and James Whitney. Quite the opposite - Im able to offer a very particular take on what I think constitutes engaging Visual Music - and for me thats become the real-time, elemental and harmonic correspondence between sound and image.

To this end Im building my own hybrid analogue/digital instrument to explore the aesthetics of cymatic patterns and forms - the Augmented Tonoscope. Im still within the first year of my PhD, my studies are at a relatively early stage, so being selected to present a paper at Understanding VISUAL Music 11 is a heartening validation of my research agenda. I plan to share key aspects of my current thinking and practical experimentation and finish with an indication of future developments and directions.

Motivation
My interest in Visual Music stems from over 30 years experience and practice as a semi-pro musician, music producer and record label partner; a commercial graphic, web and interaction designer; and an experimental, hand-drawn 16mm film enthusiast. These parallel strands converged about 10 years ago through an MA in Hypermedia Studies at the University of Westminster, London UK. Ive spent the past decade focusing my specific interests in the field by curating audiovisual experimentation for the sound, music art and technology festival, Cybersonica and pursuing my own audiovisual creative practice in artistic collaborations such as The Sancho Plan and more recently Monomatic. Its only natural then that my PhD should build on this existing experience in exploring: acoustic and visual interplay; real-time, musical interactions; design and fabrication of musical devices.

Research Aims
How far can artistic investigation into Cymatics - the study of wave phenomenon and vibration contribute towards a deeper understanding of the interplay between sound and image in Visual Music? The term Cymatics (from Greek: wave) was coined by the researcher Dr Hans Jenny (1967, 1972) who studied this subset of modal wave phenomena using a device of his own design - the tonoscope. As John Telfer (2010) explains Sound can induce visible pattern. When physical matter is vibrated with sound it adopts geometric formations. I aim to develop this line of research by exploring the aesthetics of cymatic patterns and forms. Is it possible to develop a cymatic visual equivalence to the auditory intricacies of melody, harmony and rhythm? The focus of my research is to design, fabricate and craft a sonically and visually responsive contemporary version of Jennys sound visualisation tool - the Augmented Tonoscope. I then intend to play, record and interact with it to produce a series of artistic works for live performance, screening and installation.

Approach
After 30 years of uninterrupted practice trying to push the envelope of what is possible creatively with the aid of technology, my research seems to be leading me towards fundamentals. Im drawn to a reductionist, back to basics approach to exploring the interplay between sound and image. This has manifested itself as a start from scratch methodology in the design, fabrication

and crafting of my own hybrid analogue/digital instrument. Im currently focused on prototyping a series of analogue tonoscopes driven by a custom-made Sine Wave Generator incorporating light and camera control built on the Arduino open source electronics prototyping platform. While artists have always worked with new technologies, Id argue there is a tendency in some circles towards a technological fetishism - founded on a false belief that the novel will lead to the innovative. Here I concur with Moritz (1986) and his argument about the delusion of technology - that the history of audiovisual art is littered with new inventions heralded as breakthroughs by their inventors which history shows us to be little more than technological dead-ends and eccentric curiosities. The fact that Im using micro controllers, open sound control and open-access fabrication technologies is actually not the point - they are just the tools of this time at my disposal. Id like to think Im doing more than just indulging an artistic passing fancy with the burgeoning DIY electronics movement - Im pursuing a thoughtful and pragmatic approach to problem solving by applying my own research frameworks. I think a useful insight into my general approach is illustrated through a question I was asked at EVA 11: Why not just buy a commercial sine wave generator? Why bother to build my own? My considered response being: 1. I want a custom tool designed specifically for the job - a device that behaves the way I want; 2. Like a musical instrument, Im trying to develop an intimacy with the device that will make my interaction with it more intuitive; 3. I want to engage in a design dialogue with the device - allowing it to inform and shape the design process; 4. Im trying to be somewhat systematic in locating serendipity i.e. the art of making an unsought finding - in the belief, and despite the seemingly inherent contradiction, that if I do Ill be more likely to uncover the devices latent potentialities. Of course I do aspire to audiovisual innovation, but Im pretty certain it wont be through the misguided pursuit of the novel, more likely by discovering unfound linkages between dissociated strands of research. This concurs with Barbrooks (1998) notion of technological progress - that genuine innovation only happens when inventive connections are made between seemingly unrelated but parallel technological developments - vis--vis the Telegraph was only possible with both the discovery and subsequent experience of manipulating electricity AND the ability to manufacture miles of conductive cable. Im trying to be mindful of the machine used by John Whitney Sr. in the late 70s - an Andromeda Systems LSI-11, 16-bit machine operating on programmes in Pascal. It may seem arcane and woefully underpowered compared to the computational power at our disposal now - but what Whitney did was to develop an underlying methodology for generating graphic motion sequences that is still used today. Although I hope the Augmented Tonoscope will demonstrate rare subtlety and musical expressivity Im less focused on making an audiovisual breakthrough and more interested in trying to identify, explore and apply key underlying paradigms that will inform and guide my research and ultimately my practice. I believe it is in this way that I might contribute something meaningful and lasting to the ongoing evolution of Visual Music. As John Whitney Sr. (1980) declares in Digital Harmony, the purpose is to document my own approach and propose the seminal idea of making an approach.

Practical Work To Date

Much of practical work Ive undertaken to date is described in overview in a paper I presented at EVA 11 in early July - downloadable from the BCS website - or in more depth on my online Digital Sketchbook - so Im not going to go into detail here. Rather Im going to describe the focus of my research and practice in the past couple of months.

Recent Research
In Digital Harmony: On the Complementarity of Music and Visual Art, John Whitney Sr. proposes his ideas of harmonic correspondence: This hypothesis assumes the existence of a new foundation for a new art a broader context in which Pythagorean laws of harmony operate... that the attractive and repulsive forces of harmonys consonant/dissonant patterns function outside the dominion of music. He argues that harmonys charge and discharge of tension in chordal and tonal sequences within music can be matched in visual art and particularly within computer generated animation. What intrigues me about Whitneys complementarity is the notion of a visual equivalence, described by the Pythagorean laws of harmony, to both the attractive and the repulsive forces of consonant and dissonant patterns found in music. So I wonder if there is a parallel between Whitneys dynamic motion algorithms and the static and dynamic cymatic figures and forms produced by sound? Can I match both the charge and the discharge of tensions within chordal and tonal sequences cymatically? Up to this point, experiments with my analogue tonoscope prototypes had primarily been concerned with developing a cymatic scale - a sequence of specific pitches resolved to discrete geometric visual patterns - albeit as a set of formal constituents for pattern making rather than a call to a musical framework per se. Whitneys hypothesis shifted my focus somewhat when I realised that the subtleties and nuances I was looking for with the Augmented Tonoscope might only emerge in the shift and transition between these pitches. Somewhat surprisingly, Western Music has a limited number of established terms and techniques that describe the shift from pitch to pitch - and these seem to lack nuance and detail in the specifics implementation of the effect: Portamento (plural: portamenti, a noun meaning literally "carriage" or "carrying") originated from the Italian expression "portamento della voce" (carriage of the voice), denoting from the beginning of the 17th century a vocal slide between two pitches and its emulation by members of the violin family and certain wind instruments - sometimes used interchangeably with anticipation. It is also applied to one type of glissando as well as to the "glide" function of synthesizers; glissando (plural: glissandi, abbreviated gliss.) is a glide from one pitch to another. It is an Italianized musical term derived from the French glisser, to glide. In some contexts it is distinguished from the continuous portamento. Some colloquial equivalents are slide, sweep (referring to the 'discrete glissando' effects on guitar & harp respectively), bend, or 'smear; Vibrato is a musical effect consisting of a regular, pulsating change of pitch. It is used to add expression to vocal and instrumental music. Vibrato is typically characterised in terms of two factors: the amount of pitch variation ("extent of vibrato") and the speed with which the pitch is varied ("rate of vibrato").

Bret Battey (2004) concurs, arguing in comparison to Western classical music, numerous other musical traditions place far greater emphasis on the expressive shaping of the continuum between scale steps. However, he also points out that while The scalar framework on which such music is suspended may be highly codified theoretically the detailed and highly subtle pitch curves, ornaments, and inflections of the tradition usually remain at most vaguely described absorbed through oral transmission, and musical notation proves unequal in the task of representing them. So Batteys research into how computers can effectively and convincingly render expressive melodic forms inspired by pitch continuum traditions is interesting and relevant - and his technique of Bzier curve modeling - combining an identification of critical tonal points in the performance with a simple and intuitive two-dimensional designation of the curve between those points - is likely to inform my future development in trying to implement more controllable pitch curves.

Practical Implementation
Id managed a fair degree of hands-on control with the early prototype of my DIY Sine Wave Generator (SWG) and definitely plan to develop this further through a more detailed investigation into the areas of tangible computing, embodied interaction and musical gesture. But gaining more control over the way the SWG moved between two frequencies seemed a priority, so I decided to focus on developing the automatic functionality Id already introduced - a code-based implementation of a simple sinusoidal and square wave LFO with pitch modulation. Coming across Andy Browns Arduino easing functions animation library, based on the work of Robert Penner, I wondered whether the tweening used to create more natural looking movement in computer animation - actually just a range of mathematical functions of position over time - could also provide more precise and natural control over pitch. This seemed an intriguing idea with interesting potential - to apply the principles behind these now ubiquitous animation and motion graphic techniques to sound itself - and then explore how they affect the transitional states of associated cymatic patterns and forms. So Ive been focusing on integrating Andy Browns Arduino easing functions animation library to explore how I might shape the shift between musical tones using a range of common tweening functions. Next I plan to conduct a series of experiments where I look at the effect of these different functions in shaping the shift between two pitches which each generate distinct standing wave patterns in the prototype analogue tonoscope - to see whether Robert Penners efforts to provide meaningful and predictable relationships between position and time has any impact on the subtleties and nuances in the transitional states between these cymatic modal patterns. Ive since bought a second AD9835 breakout board and planned to make a smaller, portable SWG v3 to bring with me to Understanding VISUAL Music 11 - with a single Bourns ECW Digital Contacting Encoder and the Adafruit 2.8" 18-bit color TFT Touchscreen but implementing and developing the touch screen functionality. I might have managed this had I not scuppered myself by cracking its touch screen glass and breaking this capability. Ive since attempted to re-introduce some of this lost control by using input components from the earlier prototype - but its not ideal.

Next Steps Im planning to build on the work of John Telfers (2010) Cymatic Music - an audiovisual science and music project investigating the possibilities of creating a system of visual, or rather visible music. Telfer also asks if cymatic patterns can be interpreted musically but argues that Equal Temperament - the thirteen equally spaced steps of a keyboard octave embedded in the piano and fretted string instruments which has dominated Western Music since the C18th obscures the fact that this musical structure actually has harmonic inconsistencies. Telfer asserts, It is worthwhile developing a music system with firm acoustic foundations, particularly if this system is to be used as a tool for cymatic enquiry. He responds with his theory of harmonicism - a system of proportional Just Intonation - and his work in developing the two distinct harmonic and arithmetic progressions (overtones and undertones) of the Pythagorean Lambdoid - which re-emerged in the C19th in its completed form as the Lambdoma matrix - as a practical creative resource in music. Telfers research is compelling, particularly if it does indeed take advantage of the creative cross-fertilisation between musical harmony and physical patterning though it does suggests that a correspondence between tone and cymatic pattern and form will most likely not be found within the Even Temperament of Western Music but within other musical traditions. Furthermore, Telfer favours manufacturing acoustic musical instruments to interpret the harmonicism within the Lambdoma matrix but recognises - and to my mind points to - the possibility of an electronic (and digital) approach which I plan to adopt. Also of interest is the work of composer, writer and video artist Bill Alves - one of Whitneys last collaborators before his death in 1995. Having just read John Whitney Sr.s Digital Harmony its been particularly useful to re-read his ideas and approaches summarised so succinctly by Bill Alves and appreciate his influence and legacy through Alves video work. Alves built upon the principles outlined by John Whitney Sr. - extending his differential dynamics into threedimensions and exploring new interpretations of dynamic Just Intonation. The fact that whole number proportions create these arresting patterns of visual resonance suggest a correspondence, or complementarity, to consonant musical sonorities created by whole number frequency ratios, that is, Just Intonation.... In my first video based on these principles, Hiway 70 (1997), I extended the polar coordinate curves of Whitneys Permutations to three-dimensional graphics. But the most important way in which my work was distinguished from his is that, approaching this work as a composer, I created a soundtrack in tandem with the visual composition, carefully synchronizing movement between points of tension and dissonance and points of stability and tonal consonance. I created the music entirely in Just intonation, using harmonies which were often direct analogues of the patterns of visual symmetry

Future Development
In my reading of this various research Im beginning to surmise congruence between the Cymatic Music of John Telfer, the digital harmony of John Whitney Sr., the Bzier curve modeling of Battey and the dynamic Just Intonation of Bill Alves. So Im planning to summarise their key strands of thought (along with interesting asides and digressions) as a basis for developing my own line of thought - as well as designing a series of artistic experiments where I can test these ideas out practically.

During the second year of my PhD I plan to: design the virtual system (having already identified solid starting points in the open source, creative coding of Graham Wakefields Chladni 2D & 3D MaxMSP patches (2009) and Paul Falstads Math and Physics Java Applets (2005-2009)); code a series of prototype digital tonoscopes; map and critique relevant artistic and aesthetic influences; and integrate the virtual system into the physical device. I contend my research project is unique in the field in combining the analogue and digital domains. I believe there is significant potential in the real-time, dynamic and aesthetic interplay between audio and augmented cymatic visual outputs to provide new insights and contribute towards a deeper understanding of the interplay between sound and image in Visual Music which can only be addressed by designing and building a new hybrid device. Through the Augmented Tonoscope I hope to approach something of the artful mastery of Oskar Fischingers synthetic sound production experiments - as Moritz (1976) observes: Ah, but those visuals contain formulas and gestures that communicate with us subconsciously, directly, without being appreciated or evaluated. In looking for a cymatic visual equivalence to the auditory intricacies of melody, harmony and rhythm I suspect that Im looking for something elusive and fleeting. I hope that a systematic search for the unfound - focused within the subtle nuances of the interplay of aesthetics and dynamics of the Augmented Tonoscope will allow me to catch more than a glimpse... and enable me answer my research questions.

Reference
Alves, B. (2004), Digital Harmony of Sound and Light, Computer Music Journal, Volume 29, Number 4, Winter 2005 Barbrook, R. (1998), The Hi-Tech Gift Economy, http://www.imaginaryfutures.net/2007/04/19/the-hi-tech-gift-economy-by-richard-barbrook/ [accessed 28 June 2011] Battey, B. (2004), Bzier Spline Modeling of Pitch-Continuous Melodic Expression and Ornamentation, Computer Music Journal, Winter 2004, Vol. 28, No. 4, Pages 25-39 Moritz, W. (1986), Towards an Aesthetics of Visual Music, ASIFA Canada Bulletin, 14(3), 1-3. [online at http://www.iotacenter.org/visualmusic/articles/moritz/moritz_aesthetics, [accessed 10 March 2011] Moritz, W. (1976), The Importance of Being Fischinger, Ottawa International Animated Film Festival Program, 2-6. Penner, R. (2002), Motion, Tweening, and Easing, Programming Macromedia Flash MX, Berkley: McGraw-Hill/Osborne, Pages 192-217 Telfer, J. (2010), Cymatic Music, http://www.cymaticmusic.co.uk/ [accessed 8 March 2011] Whitney, J. (1980), Digital Harmony: On the Complementarity of Music and Visual Art, Peterborough, N.H.: McGraw-Hill