Research portfolio

Scientific merits and research work

Dr. Tommi Himberg, PhD

30.10. 2018
1. My mission as a researcher
I want to understand how the mind works – focusing on the most fascinating aspects of human minds:
creativity and the ability to connect to and share with other human minds. I study how people interact,
how they influence each other, how they feel and make others feel. I measure, and I try to understand
the measurements from creative, improvised interactions, be they conversations, dance, music, or
something in between.

2. Researcher training: cross-cultural psychology, social interaction and en-

trainment
My first proper research project was my master’s thesis (2002), where I conducted a cross-cultural
experiment comparing the perception of melodic complexity in South-African and Finnish teenagers. I
was lucky to be able to travel to South Africa for field work and collecting data. Supervised by Petri
Toiviainen and Tuomas Eerola, the project was also a transformative lesson in quantitative methods
and systematic musicology. The study was published in Psychology of Music a few years later (Eerola
et al., 2006).
I did my PhD studies in Cambridge, at the Centre of Music and Science, Faculty of Music, under the
supervision of prof. Ian Cross. I was in Cambridge 2003–2007. My thesis title was “Interaction in Mu-
sical Time”; it is a monograph that I eventually completed in 2014. In my PhD studies, I conceived and
implemented a two-person tapping experiment, to extend the popular solo finger tapping paradigm,
used to investigate rhythm production, synchronisation and time-keeping. In my master’s thesis I found
differences between Finnish and South-African listeners’ perception of rhythms, so I wanted to develop
a rhythm production task to further investigate these differences. Only one paper with dyadic tapping
had been published at a time, and my PhD turned into a methods development project, to be able to
study entrainment and dyadic interaction. I taught myself to code, to run dyadic experiments, and to
analyse dyadic data (also summarised in publication #10). I gained an international network of col-
leagues and saw first-hand how a top-tier university works. I contributed to the community by e.g.
organising graduate seminars, got my first experiences in popularising research, and learned to be an
independent scholar.

3. Work at the FCoE Interdisciplinary Music Research: multimodal data, kin-

ematics, and interaction in music therapy
Before completing the thesis, I moved back to Jyväskylä and worked as an assistant (later university
teacher), and got to join the Finnish Centre of Excellence in Interdisciplinary Music Research. Being a
full-time teacher left relatively little time for research, but I conducted further research on two-person
tapping, and explored the use of motion capture in studying dyadic and group interaction. With Dr.
Marc Thompson, we conducted a study with South-African and Finnish choir singers, looking at syn-
chronicity of their dance movements. I also started a collaboration with Dr. Neta Spiro at Nordoff-
Robbins Music Therapy Centre, London, and we explored ways to study music therapy interactions.
This work resulted in a MATLAB Toolbox for analysing qualitative annotations made in ELAN, numer-
ous conference presentations, many projects and theses at Nordoff-Robbins and University of Cam-
bridge with Dr. Spiro as primary supervisor, and a journal article. With these research projects I learned
to collect and analyse multimodal datasets, and write more complex software for data analysis. The
article, data, and code for this publication are freely available online, and as such it was my first fully
open research project – now a standard. Working in the FCoE also allowed me to learn about music
and neuroscience, and supervising so many student research projects helped me acquaint myself with
the full spectrum of musicology.

2
4. Hari-group: two-person neuroscience, behavioural synchrony
In 2012 I started as a post-doctoral researcher in academician of science, academy professor Riitta
Hari’s group. I completed my PhD thesis in 2013, and graduated in 2014. I was part of the Brain2Brain-
project, funded by European Research Council Advanced Grant. The aim of the project was to develop
two-person neuroscience, concurrent measurement of the brains of two interacting participants. Spe-
cifically, the team was building MEG2MEG, a connection between two magnetoencephalography de-
vices. In MEG2MEG, two separate laboratories and their MEG devices are connected, so that two par-
ticipants, each sitting in their own MEG, can have conversations or do other simple interaction tasks
through a Skype-like video-audio link, while their brain activities are measured. Developing these in-
teractive tasks and analysing the behavioural data was my job. Analysing two-brain data is theoretically
and conceptually a novel challenge for the field of neuroscience, and the brain data analysis is ongoing.
We published behavioural data from one of these experiments.
To aid the development of these dyadic protocols, I started a project, Behavioural Synchrony where we
used e.g. motion capture to investigate dyadic interaction in behavioural settings. With the BehSynch
project, I was able to lead and develop my own research orientation. In collaboration with Dr. Melisa
Stevanovic, we investigating the use of gestures, body movement, mutual gaze, and prosodic synchrony
in dyadic decision-making. This multi-disciplinary collaboration has resulted in multiple conference
papers and an article in the esteemed Research on Language and Social Interaction. Our multimodal
dataset (audio, video, motion capture, eye-tracking, and conversation analytical annotations) has ad-
ditionally been used in two master’s thesis projects (Niinisalo, 2016, Jekunen, 2018). More information
about the BehSynch project can be found online at http://bit.ly/BehSynch. The time in Hari-group has
taught me how one of the very best research groups in the world operates, and the importance of
creativity, multi-disciplinarity, and strategic long-term planning.

5. Social eMotions: combining artistic and scientific methods

In 2014 I collected together a group of researchers and dancer-choreographers, and we got funding for
a two-year research project on bodily expression of social emotions. I served as the principal investi-
gator of this 120 000€ project, funded by Kone Foundation, that employed a core group of 5 people,
plus 4 others in supporting roles or performers. The project investigated bodily expression of emotions
and how emotions are communicated between people. We combined artistic and scientific methods
to study these phenomena in the context of contemporary dance.
Our two dancer-choreographers Johanna Nuutinen and Jarkko Lehmus first did artistic research on
emotion expression, resulting in a matrix of emotions and their motion and interaction correlates, a
useful tool for choreographers. They then choreographed a short duet that could be performed in dif-
ferent emotional states. We motion-captured 37 different versions of the choreography, dancers ex-
pressing various levels of loving and loathing each other. We analysed which kinematic features were
associated with which emotion, and conducted a perceptual study to see how the audience recognises
these different emotions. The scientific part of the project was summarised in a conference paper (Him-
berg et al. ,2017), with two more papers currently in the publication pipeline.
The main outcome of the Social eMotions project was an artistic performance, a dance piece for 2
dancers, 2 cellists and a scientist. We commissioned music (composer Jussi Lampela) for two cellos,
and two professional cellists (Ulla Lampela and Iida-Vilhelmiina Laine) conducted artistic research,
and created their matrix of musical expression of different emotions and interactions. In the perfor-
mance, dancer-cellist pairs interact through improvisations based on pre-composed loops of movement
and music. We built a system where the audience members could vote on their mobile phones, which
emotions the pairs should move towards next. The emotional interactions within and between pairs,
and the dynamic projections of the dancers’ modified movements created by graphical artist Dr Roberto
Pugliese made for a fascinating art work that has to date been performed three times, premiered in the
OuDance festival in Oulu, and then two performances in Stoa theatre in Helsinki. More info, including
a video trailer of the dance work is at http://bit.ly/SocialEmotions

3
6. ICI: mirroring, improvisation, creativity, and visiting professorship
As part of the BehSynch project, I developed a version of the mirror game, an improvisation exercise
where two participants stand facing each other, and mirror their hand movements, either with one
participant leading and the other mirroring, or the two sharing leadership. The shared leadership con-
dition produced the smoothest and best synchronised performances. This study has been presented in
conferences in 2015, with a journal article in the pipeline. The main impact of the mirror game study
was that it opened the door for me to join a multidisciplinary research group in Paris, called ICI – de
l’Improvisation Conjointe à l’interaction, or “From joint improvisation to interaction”. This project is
based in the centre of excellence Labex Arts H2H, and is led by Dr Asaf Bachrach (CNRS). In this
project, we develop protocols to study group interaction processes (entrainment, shared emotions, joint
attention, creativity, metacognition) using group movement improvisation as a testbed. The group con-
sists of scientists (cognition, neuroscience, music), philosophers, and dancers (some people tick many
of these boxes). We have developed wireless sensor systems to measure movements and physiological
data from up to 10 participants at the same time, and also developed ways to study the subjective
experiences of the participants. Combining these 1st and 3rd person accounts of the improvisations is
our primary goal. Our research ethos and approach are unique in the world and have been detailed in
(Himberg et al., 2018).
During this collaboration, I was also invited as a visiting professor to Labex Arts H2H and Université
Paris 8 in 2017.

7. Current directions: creativity, phychophysiology, group emotions

My current post-doctoral research position is in the Brain and Mind Lab in Aalto University, working
in the Academy of Finland -funded project “Neurocognition of Intergroup Processes”, with professor
Mikko Sams. We are currently working on building a version 2 of the ICI social, wireless sensor setup
here in Aalto, and exploring ways of extracting physiological variables from video feeds, use of VR and
AR in social neuroscience, and in general in developing the new science of experience research as part
of the Experience Platform in Aalto. We are currently focusing on collective and shared emotions, and
social identity.

8. External funding
My PhD was funded with a two-year grant from the Helsingin Sanomat foundation, and a fees-only
grant from Arts and Humanities Research Council. The Kone Foundation grant for the Social eMotions
project is my biggest so far, and it allowed me to be a PI of my own project for two years. Our ICI-
project is funded by a number of different French grants, some of which I’ve helped write.
I served as an external reviewer in this year’s ERC StG, and I have also reviewed grant applications for
the research foundation Flanders (FWO). These experiences and the experience of having been part in
many different applications, help me in the future efforts to attract external funding. I consider this as
one of the most important jobs for a professor.
(See my CV for more details in research merits, academic leadership, work as reviewer etc., and my
publication list for the complete list of publications)