The Music Paint Machine

Painting with music

Ph.D. researcher: Luc Nijs (luc.nijs@ugent.be) Promoter: Prof. Dr. Marc Leman

The MUSIC PAINT MACHINE is an interactive music system that allows you to make a virtual painting on a computer screen by playing music. Unlike in most interactive music systems, the acoustic musical instrument is not replaced by an electronic controller (e.g. Wii) or by sensors on the body but it is the combination of body and musical instrument that allows the musician to control the system. The various musical parameters (e.g. volume, pitch) that are changed with the musical instrument, and the movements of the musician (e.g. turning, bending, stepping on coloured pressure mat) are converted into visual parameters (e.g. thickness of a line, colour, transparency, drawing direction) and displayed in real time on the computer.

The MUSIC PAINT MACHINE is primarily an educational application that aims to contribute to an embodied understanding of music and to the development of musical creativity. The core of playing with the Music Paint Machine is the personal experience of its users. Immersed in playing with musical parameters, body movements and visualizations, they are invited to explore the possibilities of the musical instrument and to experiment with body, musical instrument and music. Accordingly, the emphasis no longer centres on cognitive control, but on the physical, auditory and visual experience of the musical parameters.

The MUSIC PAINT MACHINE goes beyond the mere encouragement to explore and experiment. Through deliberate learning modules, this application also contributes to the development of technical-instrumental and musical-artistic skills. After all, each virtual painting provides rich visual information on how the musician moves and plays the musical instrument. This allows one to optimize the setting of technical and musical goals, to be aware and understand ones own skills and to further develop these skills.