Presentation at ORCiM Seminar 2013,

20-21 February 2013, Orpheus Institute, Ghent Belgium

This is a script of the presentation I gave at THE (RE)SOUNDING EXPERIMENT ORCiM 13 Seminar - reconstructed from my presentation notes and iPhoto slideshow and with minimal tidying up. Typically, I underestimated the amount of material Id prepared and didnt manage to deliver all this within the presentation. The thumbnail guides are complete and the video of the slideshow is online at: https://vimeo.com/91432225. This PDF is embedded in the relevant post at my The Augmented Tonoscope website at: http://phd.lewissykes.info/orcim-seminar-2013/ - which also includes the abstract and additional links.

1. Merging audio-visualisation into a single process The history of Visual Music has a long and rich history, dating back to the colour wheel of Isaac Newton and a subsequent lineage of colour organs that explored the notion of mapping qualities of sound to light - most commonly pitch to hue. However, the term is coined for the first time by the Art critic Roger Fry when he writes about the abstract paintings of Kadinsky in 1912. Yet Id argue that most Visual Music is based on the principle of a separation of audio and visual production - of creating music to accompany existing moving image work, or visuals to accompany existing music. In contrast, my research is focussed on the real-time interplay between sound and image - of merging the making of music and the creation of visuals into a single workflow. I propose that if music and image are linked from the outset and then throughout all stages of the production process this will create audiovisual work of a different quality.2. A direct correspondence between sound and image

2. A direct correspondence between sound and image Michel Chion proposes a syncresis of sound and image - a natural and irrepressible perceptual fusion of sonic and visual events when they occur simultaneously in time - and much Visual Music is based on this premise. However, Im looking for a connection between what is heard and what is seen that is more direct and elementary. While there are notable examples of Visual Music that seek a close correspondence between sound and image - the synthetic sound production, drawn sound experiments of early experimental film makers Rudolph Pfenninger and Oskar Fischinger and the later digital harmony based computer-aided animations of John Whitney Sr. - in my opinion these are relatively rare. Im trying to avoid the potential shallowness of syncresis and to minimise the subjectivity of artistic interpretation - Im far more interested in the phenomenal than the noumenal. So Ive looked for a visible counterpart to audible frequency - a visual musical scale that is reproducible and consistent. Im deploying Cymatics - the study of modal standing wave phenomena and visible sound and vibration - to deploy the natural property of sound and vibration to generate visible patterns and forms on the surface of a vibrating diaphragm. As a result, the relationship between the sonic and visual elements is as analogs of each other in aural and visual form - essentially two different ways of sensing the same thing. I suspect that this approach may create audiovisual works in which there is a co-sensing of a co-expressiveness between aural and visual components intimately linked through the natural processes of Cymatics and resulting in a subtly shifted audiovisual experience.

3. Making an instrument To pursue this agenda, a key method of my research has been the design, fabrication and crafting of a hybrid analogue/digital instrument - a contemporary version of the tonoscope - the sonic visualisation tool made by Cymatics researcher, Hans Jenny, in his seminal studies in this area during the 60s & 70s. Ive used contemporary, open source micro-controller platforms, creative coding environments and open access fabrication technologies to make my own instrument from scratch. This slow and steady, step-by-step approach has made it possible for me to look longer and more deeply at this elementary relationship between sound and image - without the distractions of more sophisticated commercial tools and software. It has enabled me to engage in a design dialogue with the device - allowing it to inform and contribute to the design process.

It has also allowed me to court serendipity - the art of making the unsought finding - by being more open to the latent potentialities within my own electronic circuits and creative code.

4. Vibrating plates and diaphragms The C18th German mathematician, Ernst Chladni, devised a technique of drawing a violin bow against a metal plate (with a handle fixed at its centre), on which he sprinkled fine sand. By drawing the bow at various rates he was able to vibrate the device into different modes of standing waves made visible as distinct patterns in the sand. He documented the figures that appeared on plates of various sizes, shapes and materials and formulated an equation, Chladnis Law, to describe the effect. In his experiments, Hans Jenny also used metal plates, albeit excited by crystal oscillators, but his tonoscope used a rubber diaphragm stretched over a tube. I spent a fair amount of time experimenting with different tonoscope prototypes employing a range of speakers and transducers and plates and diaphragms of various materials. A particular challenge was trying to devise a dependable method for tensioning latex rubber over an acrylic dome - until it was pointed out to me by my Director of Studies that I didnt need to reinvent the drum. So my latest prototype adapts a 13 piccolo snare by removing the bottom skin and fixing a Faital Pro 12 speaker in its place. This drum and speaker assembly is then mounted into the shell of a 13 tom. I tested a range of commercial drum skins of different types and settled on a relatively heavy coated drum head which seemed to produce the best cymatic effects. With this configuration Ive been able to generate distinct cymatic patterns over a frequency range between 170-600Hz.

5. A cymatic musical scale? While all my early tonoscope prototypes displayed the dynamic, periodic and oscillatory forms of cymatic systems described by Hans Jenny - shifting shapes, pulsating globules and sometimes quite violent volcanic like eruptions of the fine reflective glass beads I used as an alternative to sand - it did take a while to achieve discreet modal standing waves patterns.

Once realised, I focussed on trying to find a visible musical scale, logging those frequencies which produced discreet cymatic patterns on the drum skin and then programming these tones back into the my tone generator so it could play this scale of notes and patterns. Yet I quickly became less interested in this simple correlation of musical tone to visible pattern once I noticed that significant differences in the transitory states between discreet patterns could be effected by adjusting the way the device shifted from pitch to pitch. What seemed most interesting here wasnt in fact the resolution of the discreet cymatic pattern - but like the dissonance and consonance that builds tension and release within music, the transition from one distinct pattern to another through divergence and chaos.

6. Cymatics and harmonics Hans Jennys research into Cymatics has inspired a range of contemporary artistic works that utilise the effects and manifestations of sound and vibration in their making and outputs, yet there is limited academic research that documents the potential of exploiting Cymatics as a means to create a visible counterpart to music. However, the Cymatic Music of John Telfer - an audiovisual science and music project that investigates the possibilities of creating a system of visual, or rather visible music - does provide much food for thought. Through his practical experiments, Telfer demonstrates that low frequency sound - in the range 12-30Hz - creates distinct, dynamic, symmetrical patterns on the surface of a bowl of water sitting on top of a speaker. He claims that these patterns show a distinct harmonic underpinning in their multifold symmetries, though he fails to prove a true harmonic link between frequency and pattern. I have actually managed to demonstrate a connection between cymatic pattern and arithmetic series but more of this later.

7. Perfect Intervals Telfer proposes that the Equal Temperament tuning system employed in Western Music since the C18th is actually mathematically unharmonious. While there may be have been good reason for its adoption - overcoming the limitation of the discordant Wolf Fifth in Pythagorean based tuning systems and more significantly the ability to change key without retuning the instrument - he argues that if a correspondence between musical note and cymatic pattern is to be found, it is more likely to be in perfect interval, Just Intonation based music traditions. Telfer has applied this idea by developing his theory of Harmonicism in which he revisits the Pythagorean Lamdoid - integrating the familiar harmonic progression of overtones - perfect intervals based on the whole number ratio sequence of 1/1, 2/1, 3/1 etc. - with the lesser known arithmetic progression of undertones - perfect intervals based on the whole number ratio sequence of 1/1, 1/2, 1/3 etc. By making these two progressions the axis of a 2D grid he has developed a practical, creative resource for music making in his Lamdoma Matrix. Each cell in the matrix represents a ratio defined by its position in the grid i.e. a cell in the third column and fourth row has a ratio of 3/4 or 0.75 of the unity, 1/1, base frequency - 330Hz if the base frequency is 440Hz. Telfer has also attempted a colour coding of the matrix providing a mapping of relative consonances and dissonances and revealing its underlying symmetrical structure.

8. A harmonic interface While Telfer prefers exploring the musical potentialities of his matrix by building acoustic stringed instruments, Ive adapted it as the basis for a musical interface for the Augmented Tonoscope. I was struck by the similarity between the 16-limit, 256 cell Lamdoma Matrix and the button grid of the monome.org minimalist controllers. So I coded an application that turned my grayscale 64 into a physical window onto the matrix. The buttons of the monome become the keys of a 2 dimensional keyboard that trigger a note in a custom-coded software synthesiser at a frequency defined by the cell that lies below it . Utilising the decoupled LEDs of the monome Ive also been able to map a pseudo-monochromatic version of Telfers colour coding back onto the physical device - allowing me to see something of the harmonic structure of the matrix and thus realise real-time performance.

[demo this] However, to explore the harmonic landscape of this tuning system based on whole number ratios more concertedly I need a mechanism for creating composition.

9. Towards cymatic composition Ive discovered a family of nodal sequencers that rethink the standard approach of the linear musical score or sequencer timeline with their vertical aspects of harmony and horizontal aspects of melody and rhythm. Drawing on these as inspiration, Im building a compositional tool that allows me to record refrains not as notes in a sequence but rather as a series of interconnected nodes. Each node has a pitch (defined by the Lamdoma Matrix via a monome button press) and is joined to its immediate neighbours by connections - the length of which represents the time between consecutive button depresses. Chains of nodes can form closed loops where melodies play repeatedly - but by leaving the chain open and by linking a node to multiple neighbours they can also form one-shot responses, scalar cascades and polyrhythmic patterns. I believe this system has the potential to create complex yet dynamic musical compositions somewhere in the interesting middle ground between authored commercial sequencers and generative music apps.

10. Harmonic relationships within cymatic patterns So far Ive tried to shape this presentation about my research into a linear narrative - even though its more frequently tangential. So Im closing this presentation with a few interesting asides. I mentioned earlier that Id managed to demonstrate a connection between cymatic pattern and arithmetic series. As I programmed more functionality into my tone generating device and I was able to ask more sophisticated demands of it. So I started to explore the harmonic relationship between cymatic patterns for myself - by selecting a frequency on one oscillator and then playing the first ten values in the arithmetic progression of undertones for that frequency on a second oscillator - i.e. frequencies based on the whole number ratio series of 1/2, 1/3, 1/4 etc. of the root. What I discovered was that if I chose a root frequency that generated a distinct cymatic pattern on the drum skin - a well defined modal standing wave - then all the subsequent frequencies in the arithmetic series also generated a distinct pattern - down to about a ration of 1/8 after which the differences between successive frequencies becomes too small.

For me this suggests a strong correlation between frequency, undertones and cymatic pattern that Im continuing to explore. 11. Portamento to Bezier curves Earlier I spoke of my interest in the transition from one Cymatic pattern to another by controlling the way the device shifted from pitch to pitch. This area seemed worthy of further investigation. Western music has a limited number of terms and techniques to describe the shift from pitch to pitch - essentially portamento and glissando - and these seem to lack detail in the specific implementation of the effect. By comparison to Western Music, numerous other musical traditions - for example South Indian vocal music - place far greater emphasis on the expressive shaping of the continuum between scale steps - but the detailed and highly subtle pitch curves, ornaments, and inflections of the tradition usually remain at most vaguely described absorbed through oral transmission. Looking elsewhere I came across Robert Penners tweening functions - well established techniques to describe the precise shift from point to point within motion graphics. These mathematical algorithms define a scalar interpolation from one position to another, a numerical change in position over time. In fact they determine position as a function of time. This made me think about whether frequency could be thought of in the same way - so that movement/music is interpolation from one position/pitch to another - a position/pitch for a specific time. By coding these Tweening algorithms into my sound making device I was able to generate a relatively sophisticated shaping of the shift from pitch to pitch. But despite being a useful starting point, Robert Penners functions are still essentially presets with minimal user control - and this led me to search further. Bret Batteys research into how computers can effectively and convincingly render expressive melodic forms inspired by pitch continuum traditions and his technique of Bzier curve modelling - combining an identification of critical tonal points in the performance with a simple and intuitive two-dimensional designation of the curve between those points - looked far more promising, interesting and relevant. While Batteys software is designed to analyse music recordings, it has inspired my development of a performative, realtime, touchscreen driven Bzier curve controller to realise more controllable transitions between pitch. 12. The Cymatic Adufe At the outset of my research project, I suggested that one intended output would be an installation piece for a gallery setting based on the principles of Cymatics. An opportunity to realise this came via an invitation from fellow MIRIAD researcher, Christina Rodrigues, to exhibit as part of a travelling exhibition, The 21st Century Rural Museum, a component of her Designing for Desertification PhD project. In May 2012 I travelled to Monsanto, a small, mountain top village in the Idanha a Nova region of central, rural Portugal and spent time experiencing the local culture. I was particularly struck by the Adufeiras do Monsanto, a group of elderly women from the village who performed in brightly coloured costumes and sang a repertoire of traditional folk songs accompanied by rhythms played on an adufe - a square, framed drum of Arabic origin.

My response was to conceive of and build the Cymatic Adufe - a technological, sculptural, sound-based, generative, audiovisual artwork that explores the rich vocal and musical tradition of the Idanha a Nova region. The work investigates the interplay between sounds and images, materials and forms emblematic of rural life. It deploys Cymatics to visualise the traditional Portuguese folk melody of the Senhora do Almorto as dynamic and shifting patterns on the surface of an adufe. Simultaneously projected onto the top of the adufe, superimposing on and augmenting these natural cymatics forms, are digital versions of geometric patterns often found in rural Portuguese decorative design and architectural elements - generated digitally using a sound responsive adaptation of the superformula, a generalisation of the superellipse first proposed by Johan Gielis which he suggests can be used to describe many complex shapes and curves found in nature. The work attempts to show how the haunting melody of the Senhora do Almurto not only has an analog in visual form - seen through the wondrous Cymatics effects of sound and vibration - but that these forms show a correspondence to traditional geometric patterns of decorative design.

13. In conclusion... Ive always appreciated that music is more than just a sequence of notes its emotional power comes from its ability to embed complex and expressive information within its very structure and form. So Ive realised that if Im going to craft an audiovisual instrument that is able to capture something of this quality in audiovisual composition then I have to prioritise an engagement with subtlety, an accent on nuance and an interest in the fleeting in to its outputs. I hope that my presentation has gone some way to demonstrate that I am on the right track.