Computational Methods in

Neuroimaging
Krishna Prasad Miyapuram
Cognitive Science & Computer Science
Indian Institute of Technology Gandhinagar

NUiCONE 2012

kprasad@iitgn.ac.in
Journey of a thousand miles begins with a single step

2012
M.Tech. 2011
2008
M.Sc. 2004 Predictive coding
2002
2000
1998 Electronics
Artificial Intelligence Neuroeconomics
Cognitive Neuroscience
 Outline
• Imaging the Human Brain
• Physics of Functional MRI
• Classical analysis: Statistical Parametric
Mapping
• Data Visualization
• Beyond Blobs: Functional Connectivity
• Machine Learning Methods
• Data Mining Techniques
 The BIG Question
What is the nature of
human MIND? Psychology

Artificial
Philosophy
Intelligence

Cognitive
Anthropology Science Neuroscience

Linguistics Education
 The small Answer
Study the human
BRAIN!
Neuroimaging Techniques
Parts of the Brain
Frontal Lobe
Parietal Lobe

Occipital Lobe
Temporal Lobe
 Terminology – Planes and Sections
Coronal

Axial

Coronal Plane
Axial /
Horizontal
Saggital Plane

Saggital Plane
3D imaging
Sectional view of an MRI Scanner

Scanner room

Static magnetic field

Gradient coil

RF (Head) coil

The Nobel Prize in Physiology or Medicine for

Patient Table 2003 jointly to

Console room

Paul C. Lauterbur Sir Peter Mansfield

"for their discoveries concerning

magnetic resonance imaging“
http://www.nobel.se/medicine/laureates/2003/press.html
 Physics of MRI
(A) (B) Rest (Normal blood flow)
Brain activity

Oxygen consumption Activation (High blood flow)

Cerebral blood flow

Oxyhaemoglobin (C)
Deoxyhaemoglobin

MRI signal intensity

Experimental Design
Task condition Baseline
Input
Input

Process
Output

Output

A A A A A A
B B B B B B B B

A B A B B A B A A B A
 Data Analysis
Statistical Analysis
fMRI General Linear Model
time series Design matrix
Parameter estimation

Within-subject registration Statistical Inference

slice-timing correction Linear Contrasts
Realignment Thresholding
Coregistration
(structural to functional) Random Effects Analysis
(Group analysis only)
Between-subject registration
spatial normalization
Statistical
Spatial smoothing
Parametric Map

Preprocessing
Softwares for fMRI Analysis
 Statistical Parametric Mapping
• SPM is a form of data
reduction, condensing
information (in a
statistically meaningful
way) from a number
of individual scans into
a single image volume
that can be more
easily viewed and
interpreted.
SPM has an extensive web site at:
http://www.fil.ion.ucl.ac.uk/spm
 Image Processing

Within-subject registration
slice-timing correction
Realignment
Coregistration
(structural to functional)

Between-subject registration
spatial normalization

Spatial smoothing
 Need for motion correction
• People move, even if they don’t realize!
Same location in the brain

Same location in the grid

Rigid body movement: 3 translation parameters (A) (B)

3 rotation parameters
z

roll
pitch (C) (D)
x

y yaw
 Realignment
Two steps:
Registration: Determine the 6 parameters that describe the rigid-body
transformation between each image and a reference image (usu. first in series).
Transformation: Resampling each image according to the determined
transformation parameters.
Small movements are corrected well

TRANSLATION
x
mm

y
z

ROTATION

pitch (x)
rad

yaw (y)
roll (z)

Sudden movements are more problematic

(especially if correlated with experimental paradigm)
Intra-volume time acquisition correction (slice timing)
 Slice Acquisition

Interleaved sequence

Ascending sequence
 Structural differences between subjects: Normalization

Different People Different Brains!!

Also useful for reporting coordinates in a standard space (e.g., Talairach and Tournoux)
 Smoothing

Why smooth?
• Remove residual inter-subject brain differences
• Allow for the use of Gaussian random field theory (later…)
Statistical Analysis

General Linear Model

Design matrix
Parameter estimation

Statistical Inference
Linear Contrasts
Thresholding

Random Effects Analysis

(Group analysis only)
 Statistics: How?
fMRI model setup

• A General Linear Model

(GLM) is setup modelling
the control and test
conditions as effects of
interest.

y = Xb + e
• The GLM is used to specify
the conditions in the form
of a design matrix, which
defines the experimental
design and the nature of
hypothesis testing to be
implemented.
 Specifying Contrasts
– A contrast can be used to compare different
conditions in the study.
– The conditions that are of interest are given a
positive value, such as 1, and conditions that
are subtracted from the conditions of interest
are given a negative value, such as -1.

Thresholding:
During the assessment of
Results, height and extent thresholds are
applied to determine significant activations.
 Visualization
Glass Brain for Active-Rest Brain Slice picture for Active-Rest
 Rendering onto Subject’s Anatomical Brain
• A High resolution
anatomical image
(dimensions: 128x128x160
, resolution 1.95 x 1.95 x 1
mm) is acquired.
• This image is Segmented
into Grey, White and CSF
images.
• The subjects brain is
extracted from the Grey
and White matter images.
• The activations can now
be rendered onto 3D
anatomical image of the
subject
 Psychophysiological Interactions
• Slides from Roland Benoit, MfD 2007/8
• Data from
– C. Buchel and K. Friston. Modulation of connectivity in
visual pathways by attention: Cortical interactions
evaluated with structural equation modelling and
fMRI, Cerebral Cortex, 7: 768-778, 1997
• Figures from
– K.J. Friston, C. Buchel, G.R. Fink, J. Morris, E. Rolls, and
R. Dolan. Psychophysiological and modulatory
interactions in Neuroimaging. NeuroImage, 6:218-229,
1997
– Christian Ruff’s ppt “Experimental Design”
• Tutorial: http://www.fil.ion.ucl.ac.uk/spm/data/
 Functional Connectivity

Functional Segregation Functional Integration

Functional Connectivity Effective Connectivity

An Example
Attention

V1
V5
Two Interpretations

Context-sensitive connectivity Modulation of stimulus-specific responses

Set stimuli

source source
target target
How it works: Interactions

V1 X Attention
How it is done: PPI & SPM5
• Estimate GLM
• Extract time series at Region of Interest
How it is done: PPI & SPM5
3. Deconvolve, Calculate Interaction, Reconvolve
How it is done: PPI & SPM5
3. Estimate new GLM
How it works: GLM

0 0 1 z = -9 mm

V1 Att V1XAtt
Multi Voxel Pattern Analysis
 Problem Statement
• Over the past decade functional Magnetic Resonance Imaging (fMRI) has
emerged as a powerful technique to locate activity of human brain while
engaged in a particular task or cognitive state.

• We consider the inverse problem of detecting the cognitive state of a

human subject based on the fMRI data.
• f : fMRI-sequence(t1,tN)  CognitiveState

f
fMRI SCAN 1 fMRI SCAN 2 fMRI SCAN N Dominant Cognitive State

fMRI time series (N >= 1)

 Definition and Motivation
• What is Cognitive State?

• It is the state of a process/operation within the human

brain that affects it’s mental contents.

• Motivation :

– Such functions could provide the basis for a new approach

– to study human reasoning processes.

– Also deeper understanding of the functioning of human

– brain could help us build more advanced AI systems.
 Visuo Motor Sequence Learning
Visuo-Motor Mapping:
S
• Association of various visual e
Visual
Instruction 1
Motor Action 1
instructions to appropriate q
u
actions. e
Visual
• -Stopping at red traffic signal n
c
Instruction 2
Motor Action 2

• -Driving slowly at speed e

• breaker L
Visual
Instruction 3 Motor Action 3
e
a
Sequence Learning: r
n
• Learning a task that requires i
sequencing a number of n
g
Visual
Instruction N
Motor Action N
actions to achieve a goal.
• -Driving a car
• -Lacing a shoe
Visuo-Motor Mappings
 Visuo-Motor Tasks
Visual Display Keypad Response Visual Display Keypad Response

1 2
2 1
P P
o o 1
s 1 s 2
i 2 i
t t
i i
o 1 o 2
n 2 n 1

1
S S
1 2
e e
2
q q
u u
1
e e
n n 2 1
c 2 c
e 2 e 2
1
1

Position-to-Position Mapping Position-to-Color Mapping

 Classification Problem
P2P Vs P2C: Detect the
following cognitive states Visual P2C Arbitrary
Mapping
– “subject is paying attention Stimuli
(Position-to-Color)
only towards the position
of the visual stimuli”
P2P
– “subject is paying attention Response
towards the position and
color of the visual stimuli”

Early Vs Late Learning: Detect

– the following cognitive Visual Trial & Error
states Stimuli

– “subject has learnt the V-M Early

sequence” Late Learning
– “subject is in the early Learning

process of learning the V- Response

M sequence”
 Machine Learning Approach
• To estimate the function f: fMRI-sequence(t1, tN) ->
CognitiveState we have explored the following machine
learning techniques:

 Gaussian Naïve Bayes (GNB) Classifier

 k-Nearest Neighbor (kNN) Algorithm
 Support Vector Machines (SVM)

• Single-Subject Classifier: Classifiers that are trained and

tested with a single subject’s fMRI data.

• Multiple-Subject Classifier: Classifiers that are trained with

fMRI data of multiple subjects and tested with data of a
new subject,
 Major Challenges

3. Variation in the level of

1. Very high dimensional data
fMRI activity across
(184707 voxels/features).
subjects.
2. Variation in shapes and
sizes of brain across human
subjects.
 Feature Selection
• Select the n most discriminating voxels (Discrim) :
Voxels are selected based on their ability to distinguish
one target class (Cognitive States) from the other.
• – Select the n most active voxels (Active) : Voxels are
selected based on their ability to distinguish either
target class (Cognitive States) from the baseline
condition.
• – Select the n feature pairs whose correlation
discriminates the target classes (CorrPair) : Voxel pairs
are selected based on the ability of their correlation to
discriminate the target classes.
• We observed Poor performance of Discrim and Active
features and relatively better performance of CorrPair
features for multiple-subject classifiers.
 P2P Vs P2C Classification Study
Classification Accuracy (%)
100
Single Subject

96 Discrim
92 Active
Feature
88 Selection
50 100 200 50 100 200 50 100 200 50 100 200
GNB KNN (k=5) KNN (k=9) SVM
No. of features

CorrPair Feature Selection No. of

Multiple Subject

80
features
60 200
40 474

20
t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72
GNB KNN (k=5) KNN (k=9) SVM
Time Interval
 Early Vs Late Learning Study
Classification Accuracy (%)
100

97
Single Subject

85
Feature
Selection
70
Discrim
55 Active

40
50 100 200 50 100 200 50 100 200 50 100 200
GNB KNN (k=5) KNN (k=9) SVM
No. of features
CorrPair Feature Selection No. of
80
Multiple Subject

features
60 200
40 445

20
t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72 t=12 t=36 t=72
GNB KNN (k=5) KNN (k=9) SVM
Time Interval
 Interim Conclusions
• The problem of detection of cognitive states in such a high
dimensional feature space is feasible when right choice of
features is made along with suitable methods for
representation of data.

• Overall much better performance of single-subject

classifiers over the multiple-subject classifiers.
• We were unsuccessful in learning a classifier function for
“Four-Way Classification Study", the question that we can
detect all the cognitive states is yet to be answered.

P2C P2C Early P2C Late

P2P P2P Early P2P Late

Early Late
 Imagery  Conditioning
• Imagery: Mental States like those that arise
during perception but occur in the absence of
immediate sensory input.
• What occurs in your mind when you see the
following word
 Imagery  Conditioning
• Were your mental • Kosslyn
contents like this
– Has shape (round)
– Has colour (brown)
– Is type (cake)

OR
Pylyshyn

Neither of the above …

OR
Umm! It’s yummy!
 Experimental Paradigm
• 2 sec
1 sec
+ 3 sec

What did you See?

Poisson ITI Scrambled Money
Nothing
(mean 4 sec) Picture Bill

Perception Imagination
Reward

No Reward
Reward Predicting Responses

Activation in Midbrain is greater

for stimuli predicting reward
than control stimuli irrespective
of the reward being Perceived
or Imagined
 Predicting the imagined contents
• Support Vector Machine classifier trained on midbrain
activation from visually presented trials successfully predicts
whether the participant is imagining a reward or control
picture
Midbrain activation (cluster average)

Visual presentation  CS+  CS-

Imagination  CS+  CS-

Participants
Data mining: Meta Analysis
 Summary
• Computational Tools are indispensible for
neuroimaging
• Classical Analysis uses a standard framework
for functional localization
• We can ask questions about functional
Integration (a.k.a. effective connectivity)
• Machine Learning Methods have made the
reverse inference of cognitive states possible
• Further advances in Computational data
mining techniques are to bring in a revolution
in Neuroinformatics
 Thank you
http://cogs.iitgn.ac.in