Inter Subject Correlation of Brain Activity

during Visuo-Motor Sequence Learning

Krishna Prasad Miyapuram1, Ujjval Pamnani1, Kenji Doya2, and Raju S. Bapi3
1
Cognitive Science Program, Indian Institute of Technology, Gandhinagar, India
{kprasad,ujjval.pamnani}@iitgn.ac.in
2
Neural Computation Unit, Okinawa Institute of Science and Technology, Okinawa, Japan
doya@oist.jp
3
Center for Neural & Cognitive Sciences,
Department of Computer and Information Sciences, University of Hyderabad, India
bapics@uohyd.ernet.in

Abstract. Brain imaging using functional MRI allows us to understand brain

function while participants are engaged in meaningful tasks. Traditionally the
experimental paradigms have been limited to repeated presentation of stimuli to
participants followed by a model-based analysis of the data. The Inter Subject
Correlation (ISC) analysis allows a model-free analysis while participants are
presented with naturalistic stimuli such as watching a movie. We extend the
ISC approach to a learning paradigm in which participants are repeatedly per-
forming a motor sequence in response to visual stimuli. We qualitatively com-
pare the correlation results across learning sessions. The preliminary result we
observe is shift of correlation activity in cerebellum across sessions. A model-
based analysis identifying task related activity compared to baseline is also re-
ported.

Keywords: inter subject correlation analysis, visuomotor, sequence learning.

1 Introduction

Functional magnetic resonance imaging (fMRI) allows us to measure brain activa-

tions corresponding to specific cognitive phenomena while participants are engaged
in a particular task [1]. Design and analysis of such experiments are based on the
cognitive subtraction technique, i.e. task-related activation is typically identified by
comparing against activation in a baseline condition. The corresponding analysis of
fMRI data is a model-based technique. For example, a general linear model (GLM)
approach can be used to specify the task conditions that the participant was presented.
For detecting the brain activation, i.e. to improve the signal to noise ratio, fMRI expe-
riments require the experimental trials of the task to be repeated a number of times. It
is assumed that the repeated trials are similar to each other. Hence in the parlance of
signal processing, averaging over multiple experimental trials would yield a good
fMRI signal. Further, in order to generalize the inferences from brain imaging expe-
riments to the population, the experiments are collected over a number of subjects.

C.K. Loo et al. (Eds.): ICONIP 2014, Part I, LNCS 8834, pp. 35–41, 2014.
© Springer International Publishing Switzerland 2014
36 K.P. Miyapuram et al.

A random effects analysis with the subjects as a random factor is used to make infe-
rences.
There exist several model-free methods for analyzing fMRI data, such as the Inde-
pendent Component Analysis (ICA) [2]. A more recently developed data driven me-
thod is the Inter-Subject Correlation (ISC) analysis [3]. Inter subject correlation (ISC)
analysis aims to quantify to what extent brains of different individuals operate in a
similar manner [3]. Previous studies in ISC analysis have used naturalistic stimuli
identically across participants. These paradigms are not amenable to a model-based
analysis such as the General Linear Model because of the complex nature of the sti-
muli and the experimental conditions are not repeatedly presented as typically done in
fMRI experiments. We present a model-free analysis using Inter-Subject Correlation
in a visuo-motor sequence learning task [4, 5]. Because ISC is a model-free approach,
it has been applied to naturalistic stimuli such as watching a dance [6], watch a movie
[7], spoken & written narratives [8], speech comprehension [9], real-life risk commu-
nication [10], action observation [11] etc. Through ISC analysis, we can find shared
hemodynamic activity in the brain across subjects during the experimental task. Basi-
cally, it finds correlation coefficients between fMRI time series of the participants in
corresponding brain regions. One study [12] investigated intra-subject correlations by
repeatedly presenting stimuli in order to test the reliability of hemodynamic activity in
natural viewing. Pajula et al. [13] have validated the ISC approach with that of a sti-
mulus – model based analysis and found the same foci of hemodynamic activity.
Moreover, it may also give us a cursory look of co-activation in different brain re-
gions while performing the task.
The present research extends ISC analysis for analyzing a block-design fMRI ex-
periment in which the participants repeatedly performed visuo-motor sequence learn-
ing. When analyzing tasks that involve learning, the experimental trials are all not
similar to each other. Hence, the assumptions of the GLM do not strictly hold for
learning paradigms. It is also known that the corresponding brain activity would shift
between different regions as the learning progresses. The inter-session differences in
such tasks can be interpreted in the context of learning-related changes in brain acti-
vation.

2 Design of Experiment

Eight participants performed a visuomotor sequence learning experiment [5]. The

Task condition required participants to learn, by trial and error, the correct order of
pressing two keys corresponding to two colored circles presented simultaneously on
the screen. Six such sets were presented. There were four possible colors – red, green,
blue, and yellow. The stimuli could appear in four possible positions – up, down, left,
right. The responses were recorded on a keyboard with similar spatial configuration.
The order of keys depended on the color of the stimuli, which remained fixed
throughout the experiment. The positions at which the stimuli were presented was
randomized every trial (see Fig. 1). The response was made depending on the position
of the stimuli. An example is given here. One set of stimuli containing blue and red
 Inter Subject Correlation of Brain Activity during Visuo-Motor Sequence Learning 37

circles is presented simultaneously at the left and bottom positions on the screen.
Participants have to discover the correct order of these two stimuli by trial and error.
Let’s say the correct order was blue followed by red. Now the participant successively
presses the two buttons corresponding to the positions at which the blue and red sti-
muli were presented i.e. left followed by bottom keys. This is done for six such sets.
If an error is made, then a flash appears on the screen and participants repeat the se-
quence from the first set. Upon completion of the sequence, the trial is repeated in a
block until fixed duration of 36 sec. There were a total of four learning sessions, each
with six blocks of sequence task. The baseline condition was alternating with the
sequence task, in which one colored circle was presented randomly at one of the four
positions. The participants had to simply press the corresponding button on the key-
pad. There were a total of seven baseline blocks per session each of duration 18 sec
and every session began and ended with the baseline condition.

Fig. 1. Color Sequence Learning Task. In one trial, a set of two colored circles blue and red are
presented on the screen. In another trial, the same set is displayed at a different position. Partic-
ipants press the button corresponding to blue circle first followed by the button corresponding
to the red circle using a keypad with four keys placed in the same spatial configuration.

3 Data Collection and Analysis

Functional images were collected in a 1.5 T whole-body scanner (Shimadzu-Markoni

Magnex Eclipse). A time series of 228 whole-brain scans were obtained for each ex-
periment composed of four sessions. In each scan, a set of 50 axial T2* weighted
gradient-echo echo-planar images[repetition time (TR) 6000 ms, echo time (TE) 55
ms, Flip angle (FA) 90, matrix 64 × 64, Field of view (FOV) 192 × 192 mm and slice
thickness of 3 mm] covering the whole-brain were collected parallel to the anterior
commissure - posterior commissure (AC-PC) line. In addition, a high-resolution T1-
weighted anatomical brain image consisting of 191 sagittal slices (TR 12 ms, TE 4.5
ms, FA 20, matrix 256×256, FOV 256×256 mm and slice thickness of 1 mm) was
collected for each subject. The ethics committee of the Brain Activity Imaging Center
(BAIC), Advanced Telecommunications Research Institute International (ATR), Kyo-
to, Japan approved the experimental protocol.
38 K.P. Miyapuram et al.

Images were preprocessed with SPM8 [14]. The preprocessing for each subject
was done using the following procedure. Images were corrected for head movements
(realignment). The normalization of images to template was done using the following
procedure. First, the structural image was coregistered with the first functional image.
The structural image was used to calculate the normalization parameters after seg-
mentation into gray matter, white matter and cerebro-spinal fluid volumes. These
normalization parameters were applied for all functional images. An isometric 3D
Gaussian kernel with a full-width at half maximum of 8 mm was used for smoothing,
as the final step of preprocessing the fMRI data.
We use the toolbox for inter-subject correlation analysis of fMRI developed by
Kauppi et. al. [15] .The four sessions of fMRI data were entered separately into a single
ISC analysis. The Pearson correlation coefficient (r) of fMRI time series was calculated
for all pairs of subjects and the average correlation coefficient was taken at every voxel.
The correlation maps were threshold at p<0.05 corrected for false discovery rate (FDR)
at full frequency band. We make a qualitative interpretation of the correlations observed
for each session in the results below. Further, we performed a model-based analysis
using statistical parametric mapping (SPM8 software) as follows. First, a general linear
model was specified identifying the onsets and durations of the sequence learning task
for each subject’s preprocessed fMRI images. The contrasts were specified to identify
brain activity for each session. The contrast maps corresponding to each of the four
sessions from all the subjects are taken to a random-effects group analysis. In this
second-level analysis, an ANOVA (Analysis of Variance) model was implemented. The
final group-level results are identified at a relatively liberal threshold of p<0.001, uncor-
rected for multiple comparisons. We present the qualitative comparison of the two ap-
proaches for analyzing the fMRI time series.

4 Results
In the following, we use the words correlation and activation interchangeably. Inter
Subject correlations (see Fig. 2) in session 1 were found in the posterior cerebellum,
medial orbitofrontal/ ventromedial prefrontal cortex, anterior striatum, dorsolateral
prefrontal cortex, and parts of temporal, parietal and occipital lobes accompanied by
extensive correlation in cortical motor areas. In session 2, the cerebellar correlations
were found more in the anterior and dorsal regions. Other regions with high correla-
tions were medial orbitofrontal/ ventromedial prefrontal cortex, posterior portions of
dorsal striatum, frontopolar areas, and portions of temporal, parietal, and occipital
regions concentrated medially. Interestingly, the correlations in cortical motor regions
found extensively in session 1 were found to be negative in session 2. In session 3,
cerebellar activity continued to be localized in the anterior and dorsal portions. The
correlations in orbitofrontal cortex extended to lateral regions also. The ventromedial
prefrontal correlations were also present. Correlations in visual areas were more con-
centrated to medial portion. Correlations in cortical motor areas were observed more
laterally. In session 4, the cerebellar correlations were localized more towards anterior
region. The correlations in ventral frontal, temporal and parietal areas were extensive-
ly found in lateral regions. Striatum and medial prefrontal correlations were also
found. The lateral cortical activity in motor areas was persistent in session 4.
 Inter Subject Correlation of Brain Activity during Visuo-Motor Sequence Learning 39

Fig. 2. Inter subject correlations result. Axial slices of Sessions 1 to 4 of Color to Position se-
quence learning task are shown as rows with slices at Z = -40, -16, 0, 16, 36, 60 mm respective-
ly, shown in columns. Values depicted are the average correlation values (Pearson coefficient)
that survived the significance threshold of p<0.05 FDR corrected (full frequency band).

When compared to the ISC analysis, the General linear model analysis revealed
fewer locations of brain activity even at a relatively liberal threshold of p<0.001. For
a qualitative comparison we have depicted the results at the same brain slices as the
results of ISC analysis. We notice that the extent of correlation-based activity was
much larger throughout different areas of the brain compared to the model-based
analysis. The model based-analysis however did identify more specific brain regions
activated (see the sagittal slice in Fig. 3).
40 K.P. Miyapuram et al.

Fig. 3. General linear model result - axial slices of Sessions 1 to 4 of Color to Position sequence
learning task are shown as rows at the same locations as Fig. 2. Additionally a mid-sagittal
view is included with overlaid activations. Values depicted are the scores of statistical parame-
ter estimated from the GLM analysis that survived threshold of p<0.001, uncorrected for mul-
tiple comparisons.

5 Discussion and Conclusion

The present research extends a model-free approach of inter-subject correlation analy-

sis to a visuo-motor sequence learning paradigm. We qualitatively identified some
interesting shifts of correlation values in few brain regions such as the cerebellum
across different learning sessions. The correlation maps had a greater extent of activa-
tions compared to the model-based analysis. This is possibly because of the block
design of the experiment with alternating baseline and sequence learning. Future work
can aim at delineating the time series from sequence learning blocks alone for ISC
analysis. It can be extended to a qualitative comparison of the ISC approach to other
model-free approaches such as the Independent Component Analysis.
One limitation of our experimental paradigm is that the task performed required a
motor action on behalf of participants. Different participants will have different rate
of learning. Hence, unlike paradigms of naturalistic stimuli, our trial and error based
learning experiment is not identical across participants. While the model-based analy-
sis is specifically task-related activity corresponding to visuo-motor sequence learning
compared to baseline, the ISC analysis performs a time series comparison across the
entire experimental session i.e. including both baseline and task. The comparison
of the model-free and model-based approaches allows us to conclude that there is
 Inter Subject Correlation of Brain Activity during Visuo-Motor Sequence Learning 41

information spread across a number of brain regions, as revealed by the ISC analysis.
We speculate that this correlation activity represents information that perhaps is sub-
threshold in a traditional General Linear Model. With this limitation, this research
extends ISC analysis to experiments beyond naturalistic stimuli.

References
1. Huettel, S., Song, A., McCarthy, G.: Functional Magnetic Resonance Imaging 2nd edn.
Sinauer, Massachusetts (2009)
2. Brown, G., Yamada, S., Sejnowski, T.: Independent component analysis at the neural
cocktail party., 54-63 (January 2001)
3. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., Malach, R.: Intersubject Synchronization of
Cortical Activity During Natural Vision., 1634 (2004)
4. Chandrasekhar Pammi, V., Miyapuram, K., , A., Samejima, K., Bapi, R., Doya, K.:
Changing the structure of complex visuo-motor sequences selectively activates the fronto-
parietal network. NeuroImage 59 (2011)
5. Miyapuram, K. P.: Visuomotor Mappings and Sequence Learning: A Whole-Brain fMRI
Investigation. M.Tech. Thesis, University of Hyderabad, India (2004)
6. Jola, C., McAleer, P., Grosbras, M., Love, S., Morison, G., Pollick, F.: Uni- and
multisensory brain areas are synchronised across spectators when watching unedited dance
recordings., 265-84 (June 2013)
7. Kauppi, J.-P., Jääskeläinen, I., Sams, M., Tohka, J.: Inter-subject correlation of brain
hemodynamic responses during watching a movie: localization in space and frequency.
(March 2010)
8. Regev, M., Honey, C., Simony, E., Hasson, U.: Selective and Invariant Neural Responses
to Spoken and Written Narratives. The Journal of Neuroscience 33(40), 15978-15988
(October 2013)
9. Wilson, S., Molnar-Szakacs, I., Iacoboni, M.: Beyond Superior Temporal
Cortex:Intersubject Correlations in Narrative Speech Comprehension. Cerebral Cortex 18,
230-242 (January 2008)
10. Schmalzle, R., Hacker, F., Renner, B., Honey, C., Schupp, H.: Neural Correlates of Risk
Perception during Real-Life Risk Communication. The Journal of Neuroscience 33(25),
10340-10347 (June 2013)
11. Nummenmaa, L., Smirnov, D., Lahnakoski, J., Glearean, E., Jaaskelainen, I., Sams, M.,
Hari, R.: Mental Action Simulation Synchronizes Action–Observation Circuits across
Individuals. The Journal of Neuroscience 34(3), 748-757 (January 2014)
12. Hasson, U., Malach, R., Heeger, D.: Reliability of cortical activity during natural
stimulation. Trends Cogn Sci. 14(1), 40 (January 2010)
13. Pajula, J., Kauppi J-P, J.-P., Tohka, J.: Inter-Subject Correlation in fMRI: Method
Validation against Stimulus-Model Based Analysis., e41196 (2012)
14. Friston, K., Ashburner, J., Kiebel, S., Nichols, T., Penny, W.: Statistical Parametric
Mapping: The Analysis of Functional Brain Images. Academic Press (2007)
15. Kauppi, J.-P., Pajula, J., Tohka, J.: A versatile software package for inter-subject
correlation based analysis of fMRI. (2014)