01

Artist Statement
During the last decades the scientific field
of artificial intelligence has undergone a
major change in direction. The general
cognitive paradigm of constructing an
artificial humanoid thinking machine was
becoming more and more problematic. It
turned out that creating an intelligent robot
capable of communication is not just a
matter of computational power. As a direct
answer to this the 90s saw the rise of a
promising and remarkable research domain
in artificial intelligence called affective
computing. In this field, originally founded
by Prof. Dr. R. Picard at MIT, researchers are
studying innovative emotional man-machine
interactions. From these efforts a new
breed of humanoid robots including the
KISMET (first generation) and Nexi (second
generation) was developed at MIT.
The groundbreaking vision affective
computing offers has also had a profound
influence on the field of computational and
cognitive musicology. Specifically it has
renewed interest in the connections
between music and emotions. Since the
tentative studies of K. Hevner in the 1930s
this connection could only be thought of in
terms of classical psychology. But by
applying the field of affective computing a
whole new perspective opens up.
This point of view lies at the foundation of
the construction of the EMO-Synth. In
implementing the system numerous new
principles and ideas have also come to

02

surface. Two seminal ideas in this context

are the following:
+ The EMO-Synth can be seen as a brand-new

virtual tool by which the artist can extend and

complement his own creative process. By using
the system it becomes possible for the user to
understand his own personal creativity on a new
level.
+ By using the system a totally new virtual

platform arises in which the boundaries between

artist, audience and the generated artifacts are
redefined. To what level are these artifacts a
creation ofthe artist or the audience? To which
extent can we speak of artistic input when the
EMO-Synth truly participates in the creative
process?
Of course the last two statements start from
a certain perspective on the artistic process.
It is my personal point of view that art
communicates through human emotions.
Every work of art implies a certain kind of
expression that can be understood
intuitively rather than rationally. It is this
intuitive and emotionally loaded component
that the EMO-Synth tries to uncover and

simulate. The realization of the EMO-Synth

project also implied a radical break with the
old notion of static art. Doing this is not a
new idea but it was only through recent
developments on both the methodological
and technical level that the implementation
of systems like the EMO-Synth becomes
possible. Recent artificial intelligence
techniques and techniques stemming from
affective computing give a whole new
insight into both the esthetic or emotional
content of art objects.
In developing the EMO-Synth I feel it as my
personal goal that the project will redefine
and reshape part of the whole new
discussion about the place of automatically
generated art in the general artistic
discourse.
Valery Vermeulen
Producer & Founder of the EMO-Synth

03

EMO-Synth
The EMO-Synth is an interactive multimedia system where the emotional manmachine interaction plays a central role.
During performances and demonstrations
the system automatically generates and
manipulates sounds and images to direct
the user in certain predefined emotional
states. The development of the EMO-Synth
relies on different techniques stemming
from a broad range of domains including
artificial intelligence, affective computing,
psychophysiology, automatic music
composition and sound and image
generation.
In the current setup emotions are
quantitatively measured using biosensors
that
monitor
psychophysiological
parameters such as heart rate (ECG or
electrocardiogram), stress (GSR or
galvanic
skin
response)
and
electrooculogram (EOG). During operation
the EMO-Synth will subsequently generate
audiovisual artifacts and analyze the
resulting emotional impact on the user.
Using techniques stemming from artificial
intelligence like interactive genetic
programming and HMMs the EMO-Synth
will try to generate audiovisuals to bring
the user in a specific emotional state.

04

The concept of the EMO-Synth is based on

the link between the apparent rational
logical and creative process. Recent
technological and scientific experiments
have provided mounting evidence of this
link. As a consequence more and more
opportunities arise for the artist to use
scientific systems in his own creative
process. I first came into contact with
these ideas during a four-year research
project I undertook between 2001 and
2005 at the Institute for Psychoacoustics
and Electronic Music (IPEM, University of
Ghent). These were a primary inspiration
to start the EMO-Synth project.

05

Prototypes
Between 2004 and 2007 the first four
prototypes of the EMO-Synth were
developed to explore the possibilities of the
system and to design possible blueprints
for future developments.
Prototype 01 was developed for the multi
media project SKANNER (collaboration with
Belgian artist Angelo Vermeulen) that
mainly dealt with the concept of human fear.
The project was shown on various locations
including the Atomium (expo Interface,
Brussels, Belgium, 2004), W139 (Amsterdam,
Netherlands, on invitation of Belgian artist
Jan Fabre, 2004), Beursschouwburg
(Subliminal Festival, Brussels, Belgium,
2005), STUK (Artefact festival and Kulturama
2005, Leuven, Belgium, 2005), Z33 (expo Feel
the young, Hasselt, Belgium, 2004). With this
prototype
first
experiments
were
elaborated in the field of the sonification of
the psychophysiological data.
Prototype 02 was developed in 2006 for the
exposition PERSEVERANCE at the Godshuis
in Sint Laureins in Belgium. A first automatic
music generating system was implemented
with this prototype. During the exposition
an
EMO-Synth
performance
was
broadcasted live on the national Belgian
radio station Studio Brussels.

06

Prototype 03 developed new possibilities to

extend the generating capabilities of the
EMO-Synth. The music generation engine
was complemented with generated light
stimuli. Prototype 03 was shown at the expo
SPOTLIGHTS at cc Hasselt in 2007 in
Belgium.
Prototype 04 was developed between 2007
and 2008 in close collaboration with the
young Belgian curator Astrid David and the
collective Office Tamuraj. The musicgenerating engine was renewed while
constructing this prototype. Music
generation in this new prototype was mainly
based on an automatic mixing engine.
Under the supervision and lead of Astrid
David experiments were also being carried
out with different forms of presentation
and confrontations between the EMO-Synth
and other artistic disciplines. This resulted
in the so-called test-cases. Testcase 1 was
held at the artists office in 2007 in Ghent,
Testcase 2 was hosted by art space
Croxhapox (Ghent, Belgium, 2007), Testcase
3 was realised in collaboration with art
organization X=10c and was held at the art
fair Lineart 2007 (Expo Hallen, Ghent,
Belgium, 2007). Testcase 4 was held both at
art gallery One Twenty Gallery (Ghent,
Belgium) and at Muhka Media (Antwerp,
Belgium) featuring the renowned Belgian
psychotherapist Wilfried Van Craen.

Prototype 05 is the current prototype of the

EMO-Synth and has been under
development since 2009. The creation of this
prototype is being supported by the Flemish
Audiovisual fund in the seminal project
entitled Montage cinema revived by the EMOSynth. This project can be seen as a logical
sequel on the different test cases held
between 2007 and 2008. As to the technical
side Prototype 05 includes a renewed sound
generation engine which incorporates all
previous technologies and a first
implementation of an image generation
system.

07

PROTOTYPE 05
Montage cinema revived by the EMO-Synth

Introduction
The general idea behind the EMO-Synth is
to create a dynamic work of art that evolves
automatically. Using a natural evolutionary
process the computer will pursue the
creation of ideal optimized and personal
audiovisual material. The EMO-Synth thus
embraces a fasciniting new paradigm that
opens the discussion on the deprecation of
the unique character of a piece of art.
Referring to earlier art history; this is a
recurrent discussion that also appeared at
the rise of photography at the start of the
20th century. At that time the general
concern was the loss of the aura that
surrounds artwork because reproduction of
photographic material severely questioned
artistic authenticity. But as history thought
us it turned out artistic potential is not equal
to its medium.

Montage cinema revived by the EMO-Synth is a

project based around the EMO-Synth where

experimentation with the visual application
of the EMO-Synth is a central key point. The
project was a result of a series of
performances so called test cases which
Office Tamuraj elaborated between 2007
and 2008 and where I acted as curator. In
this respect Montage Cinema Revived by the
EMO-Synth could be seen as a project in
which the experience and techniques used
in these testcases are merged. Themes
explored in testcases 1 to 4 included
automatic music and image generation, the
use of color- and light dynamics and

08

psychophysiology. During this process as a

curator I safeguarded the audiovisual result
for which the film constitutes the apex
Having a background in art sciences I
looked for ways to promote the thesis that
the EMO-Synth project belongs to the
domain of audiovisual arts. In order to get
this statement in the public artistic debate
I organized dialogues and reflective
moments.
In my point of view the EMO-Synth
performances refer to developments in the
Fluxus movement of the 70s. The fact that
Valery Vermeulen visualizes heart rate
frequencies and statistical data shows a lot
of similarities to how Bernar Venet put
together very explicit seance-performances
with mathematicians and scientists from
1966 to 2006. Bernar Venet moved from
Nice to New York where he joined a group
of people with backgrounds ranging from
nuclear physics to math and music. In this
respect Office Tamuraj can identify itself to
the early group of inventors of multimedia
and the pioneers of video-art with John
Cage, Nam June Paik, Yoko Ono and Bruce
Nauman as leading figures. Among these
the work of Nam Jun Paik seems to be most
related to Vermeulens artistic approach.
Like Nam Jun Paik, Valery Vermeulen has the
talent to renew the arts on a technical,
visual and auditory level. The artist Valery
Vermeulen continuously switches between
three domains by applying scientific

research to his creative process. He

consciously puts pure mathematics in the
background to serve the artistic result.
As the development of the EMO-Synth
project has an outspoken interdisciplinary
character the collective Office Tamuraj
was founded to elaborate the various
aspects of the project. Along the way
these collaborations seemed to be seminal
for the creation of several prototypes. The
addition of programmer Stif Labarque
turned out crucial for a high end
implementation of the current prototype,
the editing of the film by Arpad Dorozmai
was essential to be able to talk of montage
cinema and the collaboration with Liard
Kranen helped to bring the EMO-Synth to
what it is today. In collaborating with
Valery Ive always pursued my ambition to
realize a work in team setting where each
member plays his own crucial role. Office
Tamuraj has tried to conquer new terrain
in the audiovisual arts with its computer
controlled work of art. The drive to

capture the human emotion by means of

music is perhaps romantic, but it could
lead to new to new artistic disciplines
where the visitor can engage in a direct
and concrete relationship. The purely
observational vanishes in favour of a real
interactive component.
Astrid David
Freelance Curator

If you want to build a ship, dont

drum up the men to gather wood,
divide the work and give orders.
I n s t e a d , t e a c h t h e m t o y e a r n fo r
the vast and endless sea.
Antoine de Saint Exupery

09

PROTOTYPE 05
Montage cinema revived by the EMO-Synth

project outline
The concept of montage cinema was
conceived by a number of Russian
filmmakers in the 1920s and its core ideas
were rooted in the Soviet revolution a couple
of years prior. Different filmmakers, such as
Vertov, Pudovkin and Kuleshov, all had quite
different ideas about what montage cinema
ought to be. Therefore the word montage,
which literally means editing, is used to
describe the Russian cinema of the 1920s. In
this project though, we started off with the
theories and views of Sergej Eisenstein who
claimed that montage was the very essence
of cinema.
His idea of montage cinema was that
filmmakers needed to compose a series of
exciting events in order to emotionally
stimulate the viewer. This meant that
individual shots couldnt just follow each
other, but had to collide. He denounced the
classical Hollywood way of editing in which
temporal and spatial continuity were
quintessential. Shots had to be opposed to
one another instead in a dialectical way. This
idea was derived from the Marxist theory in
which thesis and anti-thesis combined into
syn-thesis and thus determined the course
of history. This collision of shots could be
based on conflicts of scale, volume, motion
and speed, both within and between
different frames. These conflicts result in a
new mental concept which forms itself inside
the viewers mind. This mental concept
cannot originate from the different shots coexisting, but emerges from the overall
interaction between shots. This enabled the
director to raise abstract concepts, such as
specific emotions, in the mind of the
audience. Genuine emotions could not be felt

10

through emphatic identification with an

actor/actress playing a certain role and
acting out an emotion. True emotions could
only be felt by the viewer through the
dialectical process of images colliding and
thereby generating meaning, superseding
the sum of its parts.
A frequently used example to explain this
theory of how cinema works is the famous
scene from Eisensteins film Strike in which
we see workers who are on strike and then
get attacked by law enforcement agencies.
Subsequently we see bulls who are guided
towards
the
slaughter-house.
The
juxtaposition of these shots creates a
metaphor in the viewers mind about the
workers being treated as cattle and stirs up
rage and discord about the way the workers
are humiliated. The metaphor in this example
acts as a catalyst of emotions that could not
have been set off by merely showing images
of the workers and the reaction of the
establishment to their uprising against the
system.

Montage cinema revived by the EMO-Synth is

Montage cinema revived bythe EMO-Synth also

The main goal of Montage cinema revived by

the EMO-Synth is to explore and implement
modules for the EMO-Synth so that it can be
used in a cinematographic setting. The
general idea in line with the general ideas
and philosophy of the project is to radically
abandon the concept of a static movie.
Instead we programmed the EMO-Synth to
generate new versions of any given movie
with
the
appropriate, personalized
soundtrack. The audiovisual material is
generated in a way that maximizes the
emotional impact on the user.

Montage Cinema Revived bythe EMO-Synth is a

mainly focused on the development of

Prototype 05 of the EMO-Synth in
collaboration with the collective Office
Tamuraj. Right from the start of the
development of the EMO-Synth there were
already some ideas to use it in a movie
context. After the initial developments of the
first four prototypes funds provided by the
Flemish government (supported by the
Flemish Audiovisual Fund, www.vaf.be) now
make such a realisation possible.

denotes the implementation of our first

image generating system. It is basically an
automatic video montage system in this
case. During performances the EMO-Synth
will automatically start producing a new
montage of the given movie, while
preserving semantic content. Implementing
this first image generating system has
already proved to be crucial to the further
evolution
of the
image-generating
component of the EMO-Synth.
brand-new interactive cinema experience
that meets the expectations of the viewer
as closely as possible.

For Prototype 05 images were needed that

could provoke strong emotional states
during the performances. To meet this
requirement we decided to work with an old
classic silent movie. After some internal
debate, Office Tamuraj chose Julian Ruperts
1925 version of The phantom of the Opera to
be the footage we would work with.

11

PROTOTYPE 05
Montage cinema revived by the EMO-Synth

The Phantom Of The Opera

12

The Phantom of the Opera is a 1925 film directed by Rupert Julian. The film is about Eric, a
musical genius who hides himself because of his deformed face. After he has heard
Christine, the young opera singer, he becomes hopelessly in love with her. This is the
beginning of a tragedy. Christine is engaged to the handsome young count Raoul de Chagny
and cannot feel love for a deformed freak. Driven by extreme anger and jealousy the
phantom kidnaps the young soprano.
As most of the work of the montage cinema school is protected by legal issues we looked
in the project for alternative film material apt to being applied in the "montage cinema"
context. Extensive research lead us to The Phantom of the Opera. Even though it was made
in roughly the same period as the montage film-school flourished, it has few similarities at
first glance. But closer examination reveals that it contains all the elements to be used for
montage purposes.
On one hand there is the world of the phantom, which is dark, gloomy and driven by negative
emotions. On the other there is Christines world which is bright, happy and has overall
positive connotations of love and devotion. The emotional elements within the shots are
one-dimensional and dont really evoke complex emotions and abstract concepts in the
viewers minds. But a clever re-editing, executed by the EMO-Synth which follows the
guidelines stipulated by the director of the project, adds a new dimension to the film.
The juxtaposition of shots of both worlds will lead to the emergence of new and more
complex and abstract emotions which will be reinforced by the new soundtrack that is
composed to the slightly re-edited parts of the film. The audience will experience empathy
for the villainous character of Eric. This empathy is usually not associated to malicious
characters in films and can only be achieved by means of using montage techniques. These
elements present in The Phantom of the Opera make it the ideal film to use in a montage
context without being a montage film itself.

13

Office Tamuraj
Because the elaboration of the different aspects of the EMO-Synth requires various
skills the dynamic collective Office Tamuraj was founded in 2007. The collective includes
people from different backgrounds including programmers, musicians, designers and
people with an artistic aspiration. Our main philosophy as a dynamic collective is to
allow people to join in or drop out freely. Being a collective also implies we want to voice
a single message: Office Tamuraj aims to translate recent developments in affective
computing to the world of music and image generation. In doing so it acts as a bridge
between the world of purely academic research and the artistic world. Of course we
are aware that we are not the only players trying to attain this goal. But in gathering
people with a common spirit and state of mind we want to establish a discussion
platform for the possibilities of creating a new era where man and machine can
communicate on an emotional and artistic level. For the creation of Montage Cinema
Revived bythe EMO-Synth Office Tamuraj consists of the following people:
Valery Vermeulen Electronic musician, composer, producer and developer of new
interactive multimedia software and founder of the EMO-Synth project. He obtained a
PhD in mathematics at the University of Ghent where he conducted fundamental
research in the area of algebraic group theory and geometry (PhD in Pure Mathematics
in 2001). Between 2001 and 2005 Vermeulen worked at the Institute for Psychoacoustics
and Electronic Music (IPEM, University of Ghent) on a research project focusing on the
link between music and emotions. Meanwhile he started writing and recording music
in his own production studio. Since 2004 Vermeulen has been working on the
development of the EMO-Synth and other art projects focusing on sound and music.
Besides being involved in the EMO-Synth project Vermeulen works currently as a
statistical expert and consultant and studies music at the Royal Conservatory of Ghent.
Astrid David Astrid David Studied social-cultural work at the Social College Ghent,
art-sciences at the University of Ghent, followed by an internship at the Metropolitan
University of Manchester. She worked as cultural policy coordinator on two accepted
cultural policy plans for Tielt and Eeklo, organized the exhibition Persvrance in the
Godshuis of Sint-Laureins in 2006 and was associate at the Andr Simoens Gallery
from 2006 to 2009. More info on www.david-vzw.be.
Arpad Dorozmai Graduated in Communication Sciences from the Catholic University
of Leuven in 2004 with a dissertation entitled A filmtheoretical research on the socialcriticism in Elia Kazans films. In 2004 he entered a Master-class in Film Studies and
Visual Culture at the University of Antwerp. The following year he produced a short
documentary film entitled Kandarock". For his master degree, he worked on a short
film entitled Nowhere to hide. The subsequent two years he worked for the European
Film Festival in Brussels. He came into contact with Valery Vermeulen at Testcase 1 of
Prototype 04 and eventually became a member of Office Tamuraj.

14

Liard Kranen After graduating from Waldorf Education with a project and a paper on
DOGMA95 Liard enrolled in Economy and Arts at the Utrecht School of the Arts. He
graduated in 2006 with honors. Liard was involved in numerous projects as a producer
such as the short fiction film Hotel Paradijs from Jan Krger and the long-play fiction
De Legendarische Scheiding from Lars Trim and the documentaries Hooligans from
Nikita Sutyrin and Kresty from Allard Detiger. After his these experiences in film Liard
obtained a Master of Arts degree in the EMMA European Media Master of Arts program
of the HKU. In meantime he worked as a project manager for the Changing Societies
project of BUZA (Dutch ministry of Foreign Affairs) the HKU and VGIK (All-Russian State
Institute of Cinematography Moscow). His graduation project was called Disnified
(www.disnified.org) an online interactive documentary film about the
entertainmentization of society and the thoughts of young Europeans on this topic.
Liard became one of the members of Office Tamuraj art collective in 2007. Furthermore
Liard is working on a photographic documentary and an audiovisual installation from
the Disnified project.
Stif Labarque In 2004 Stif Labarque obtained the degree of Industrial Engineer
specialized in Information Technology. For his thesis he wrote an application that served
as an online data loader to provide access analysis for a video streaming website for
La nueva Espaa (a Spanish newspaper). Since then he has been developing several
web-based and standalone applications using Microsofts .NET technology. He was
contacted by Valery for the development and implementation of Prototype 05 of the
EMO-Synth. Through the use of cutting edge technologies he has seamlessly integrated
the new EMO-Synths core and its subsystems into one application.

Valery

Astrid

Arpad

Liard

Stif

15

16

DVD not here? Go to the media section at www.emo-synth.com

17

Montage cinema revived by the EMO-Synth

was supported by

Flemish Audiovisual Fund

www.vaf.be

Flanders Image
www.flandersimage.com

Mindmedia B.V.
www.mindmedia.nl

Montage cinema revived by the EMO-Synth

was produced by

Office Tamuraj
www.myspace.com/OfficeTamuraj

More info about the EMO-Synth

www.emo-synth.com

v.u.: Valery Vermeulen - OfficeTamuraj@gmail.com

design:

www.jon-ibe.be