Linear Systems Theory Fourier Analysis

Introduction Receptive fields and Signals as sums of sine waves

mechanisms
1-d: time series
Fourier Analysis Signals as sums of sine
fMRI signal from a voxel or ROI
waves
mean firing rate of a neuron over time
Linear, shift-invariant systems auditory stimuli
Definition
2-d: static visual image, neural image
Applied to impulses, sums of impulses
Applied to sine waves, sums of sine waves
3-d: visual motion analysis
Applications 4-d: raw fMRI data

Linear Systems Analysis Spatial Vision

Systems with signals as input and output Image representation or coding

At each stage, what information is kept and
1-d: low- and high-pass filters in electronic what is lost?
equipment, fMRI data analysis, or in sound
Image analysis
production (articulators) or audition (the
ear as a filter) Nonlinear: pattern recognition
2-d: optical blur, spatial receptive field
3-d: spatio-temporal receptive field

Receptive Field Receptive Field

A spatial receptive field is an image
In any modality: that region of the sensory
apparatus that, when stimulated, can
directly affect the firing rate of a given
neuron
Spatial vision: spatial receptive field can
be mapped in visual space or on the retina
Examples:

LGN V1 with its own Fourier transform.

 Neural Image
A spatial receptive field may also be treated as a linear
system, by assuming a dense collection of neurons with the
same receptive field translated to different locations in the
visual field:
Neural Image of a Sine Wave
For a linear, shift-invariant system such as a linear model of
a receptive field, an input sine wave results in an identical
output sine wave, except for a possible lateral shift and
scaling of contrast.

Frequency Response Frequency Response

This scaling of contrast by a linear receptive field in the This scaling of contrast by a linear receptive field in the
neural image is a function of spatial frequency determined neural image is a function of spatial frequency determined
by the shape of the receptive field. by the shape of the receptive field.
Contrast Gain

Contrast Gain
Spatial Frequency Spatial Frequency

Orientation Tuning Application Preview: SF Adaptation

If a receptive field is not circularly symmetric, the scaling of
(Blakemore & Campbell, 1969)
contrast is also a function of orientation (for a given spatial
frequency) determined by the shape of the receptive field.
Contrast Gain

Orientation
 Application Preview: SF Adaptation Summary: Linear Systems Theory
(Blakemore & Campbell, 1969)
Signals can be represented as sums of sine waves
Linear, shift-invariant systems operate independently
on each sine wave, and merely scale and shift them.
A simplified model of neurons in the visual system, the
linear receptive field, results in a neural image that is
linear and shift-invariant.
Psychophysical models of the visual system might be
built of such mechanisms.
It is therefore important to understand visual stimuli in
terms of their spatial frequency content.
The same tools can be applied to other modalities (e.g.,
audition) and other signals (EEG, MRI, MEG, etc.).

Auditory example: Pure tones Frequency and amplitude

weak 100 Hz strong 100 Hz weak 1000 Hz strong 1000 Hz

Sound pressure level

Time

1/100 s

Pure tones can be described by 3 numbers:

Frequency = rate of air pressure modulation (related to pitch)
Amplitude = sound pressure level (related to loudness)
Phase = sin vs. cosine vs. another horizontal shift

Fourier components of a square wave Fourier components of a square wave

Ampl. Freq.

1 100
Square wave

-1.3
1/3 300 f + 3f/3

0
+.4

+0 1/5 500 f + 3f/3+5f/5

+0 -.2

1/7 700 f + 3f/3+5f/5+7f/7

=0
Prev
Next
= -1.1
Fourier Synthesis Building Stimuli from Sine Waves
Fourier spectrum representation of sound

Amplitude
SPL

0 10 20 30 0 100 200 300

Time (ms) Frequency (Hz)

Fourier spectra of some sounds Fundamental frequency and harmonics

Pure tone
Amplitude

White noise

Violin note

100 1000 10,000

Frequency (Hz)

Standing waves in Flute (open pipe) harmonics

a vibrating string
 Flute (open pipe) harmonics
Other notes (shorten the pipe)

Reinforcing a harmonic (forcing a node)

Complex numbers and complex exponentials

Discrete Fourier Transform
z = a + bi = Ae i! = A "#cos ! + i sin ! $% where i [or j ] = !1

real imaginary amplitude phase N %1

part part imaginary X !"k #$ = ( x !"n #$ e %i(2& /N)kn 0 ' k ' N %1
n=0
where: b
A N %1
a = Acos ! b = Asin !
! = ( x !"n #$ (cos((2& / N)kn) + i sin((2& / N)kn))
A = a2 + b2 ! = tan"1(b / a) a real n=0

Why bother?
k is frequency in cycles/image (or cycles/signal) and is computed
( A1e
i!1
)( A2e
i! 2
) = A1A2e
i(!1 + !2 ) effectively only for frequencies zero (DC), 1, 2, ..., N/2. The
vector you get back from MATLAB (fft or fft2, inverses are ifft and
ifft2), however, continues redundantly (for real signals, that is):
amplitudes phases
multiply add 0, 1, ..., N/2-1, N/2=0, N/2-1, ..., 1.
 DFT of a Sine Wave

$ 2! kn ' 2! kn 2! kn
cycles/scan Acos & " # ) = Acos # cos + Asin # sin
time points % N ( N N

The Fourier coefficient for this frequency

of k cycles/signal is:
2! k / N a + bi = (Acos ! ) + (Asin ! )i
(! / 4 in this case)

In other words, the amplitude is A and the phase

is ! .
frequency (cycles per scan) frequency (cycles per scan)

frequency frequency

Similarity Theorem
If f (x) ! F(" )
1 #"&
then f (ax) ! F% (
a $ a'

0 8 16 24 32 -16 -8 0 8 16

If you stretch the x-axis, you shrink the frequency axis

0 8 16 24 32 -16 -8 0 8 16
time
 Circular shift

f, 90 deg shift

If x[n] X[k], then:

2f, 180 deg shift

Properties of the DFT x[(n-m)N] exp(-j2! km/N) X[k]

i.e., only phase (not magnitude) is affected

N-2 0 1 2 3 4 -6 -5 -4 -3 -2 -1 0 1 2 3 4 5
N-1
 i.e., all cosine terms

To generate a signal or image from a transform:

i.e., all sine terms

Other Properties of the Fourier Transform

Linearity f + g ! F +G

d cos 2!" x d sin 2!" x

Derivative = #2!" sin 2!" x, = 2!" cos 2!" x
x x
Hence:
df
$ #i2!" F DTFT and DFS
dx

iF
Integral ! f dx " 2#$
Discrete-Time Fourier Transform (DTFT)

Two-dimensional Fourier transform

Edge effects: vs.

 Homogeneity (scalar rule)
Linear, Shift-Invariant Systems
Neural activity fMRI response

Linearity: scalar rule and additivity

Applied to impulse, sums of impulses
Applied to sine waves, sums of sine waves

Additivity Shift invariance

Linear systems Shift invariance

A system (or transform) converts (or maps) an input signal into an
output signal:
y(t) = T[x(t)] For a system to be shift-invariant (or time-invariant) means that a
time-shifted version of the input yields a time-shifted version of the
A linear system satisfies the following properties: output:
y(t) = T[x(t)]
1) Homogeneity (scalar rule):
T(a x(t)] = a y(t) y(t - s) = T[x(t - s)]
2) Additivity:
T(x1(t) + x2(t)] = y1(t) + y2(t) The response y(t - s) is identical to the response y(t), except that it
is shifted in time.
Often, these two properties are written together and called
superposition:
T(a x1(t) + b x2(t)] = a y1(t) + b y2(t)
 Neural Image - Reprise Linear, Shift-Invariant Systems

A spatial receptive field may also be treated as a linear Linearity: Scalar rule and additivity
system, by assuming a dense collection of neurons with the
same receptive field translated to different locations in the Applied to impulse, sums of impulses
visual field. In this view, it is a linear, shift-invariant system. Applied to sine waves, sums of sine waves

Convolution as sum of impulse responses Convolution as sum of impulse responses

1 2 3 4 5 6 7 8 9 10111213 14
Input: Input S(t):

Impulse response I(t):

$1 I(t t 1 ):

Impulse response:
$2 I(t t 2 ):

$3 I(t t 3 ):

Output: $8 I(t t 8 ):

$9 I(t t 9 ):

$14 I(t t 14 ):

14
3 $i I(t ti ):
+ i1

Convolution Convolution derivation

Homogeneity:
Discrete-time signal: x[n] = [x1, x2, x3, ...] T{a x[n]} = a T{x[n]}

A system or transform maps an input signal into an output signal: Additivity:

y[n] = T{x[n]} T{x1[n] + x2[n]} = T{x1[n]} + T{x2[n]}

A shift-invariant, linear system can always be expressed as a

Superposition:
convolution:
T{a x1[n] + b x2[n]} = a T{x1[n]} + b T{x2[n]}
y[n] = 3 x[m] h[n-m]
Shift-invariance:
where h[n] is the impulse response. y[n] = T{x[n]} => y[n-m] = T{x[n-m]}
 Convolution derivation (contd.) Convolution derivation (cont)
x[n]: input
Impulse sequence: y[n] = T{x[n]}: output
d[n] = 1 for n = 0, d[n] = 0 otherwise h[n] = T{d[n]}: impulse response
Any sequence can be expressed as a sum of impulses: 1) Represent input as sum of impulses:
x[n] = 3x[m] d[n-m] y[n] = T{x[n]}
y[n] = T { 3x[m] d[n-m] }
where
d[n-m] is impulse shifted to sample m 2) Use superposition:
x[m] is the height of that impulse

Example:
y[n] = 3 x[m] T{d[n-m]}
3) Use shift-invariance:

3x[m] h[n-m]
= + +
y[n] =

Convolution as sum of impulse responses Convolution as correlation with the

receptive field (time-reversed impulse
Input S(t): 1 2 3 4 5 6 7 8 9 10111213 14
response): Input S(t): 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Impulse response I(t):

RF 1 (t):

$1 I(t t 1 ):
RF 1 (t)S(t):

$2 I(t t 2 ):
RF 2 (t):

$3 I(t t 3 ): RF 2 (t)S(t):

$8 I(t t 8 ): RF 8 (t):

$9 I(t t 9 ): RF 8 (t)S(t):

RF 9 (t):
$14 I(t t 14 ):

RF 9 (t)S(t):
14
3 $i I(t ti ):
i1

F (t):

Convolution as matrix multiplication Convolution as sequence of weighted sums

Linear system <=> matrix multiplication
Shift-invariant linear system <=> Toeplitz matrix
past present future
0 0 0 1 0 0 0 0 0 input (impulse)

1 1/8 1/4 1/2 weights

5 2
1 2 3 0 0 0 0 0 0 1/2 1/4 1/8 0 0 0 output (impulse response)
0
2 0 1 2 3 0 0
-3
= 0 0 1 2 3 0
0
-1
4 0 0 0 1 2 3 2
0 0 0 1 1 1 1 1 1 input (step)

1/8 1/4 1/2 weights

0 0 0 1/2 3/4 7/8 7/8 7/8 7/8 output (step response)

Columns contain shifted copies of the impulse response. Figure 3: Convolution as a series of weighted sums.
Rows contain time-reversed copies of impulse response.

Using homogeneity,

Now let be the response of to the unshifted unit impulse, i.e., . Then by usin
shift-invariance,
 discrete-time
In what
It is follows, sequences
we areconvenient
will express
often mathematically the only
most ofthings that can
withbecontinuous-time
thetomathematics
work stored
in and computed
the continuous-time with computers.
domain.
signals. But inBut
practice,
the you
examples
usuallywill, byup
end necessity,
with use discrete-time
discrete-time sequences.
sequences because: (1) discrete-time samples are the only
In what follows, we will express most of the mathematics in the continuous-time domain. But
things
the that can be
examples measured
will, and recorded
by necessity, when doing
use discrete-time a real experiment; and (2) finite-length,
sequences.
discrete-time sequences are the only things that can be stored and computed with computers.
Pulse and impulse signals. The unit impulse signal, written , is one at , and zero
everywhere else:follows, we will express most of the mathematics in the continuous-time domain. But
In what
Pulse and impulse signals. The unit impulse signal, written , is one at , and zero
Pulses and impulses the examples will, by necessity, use discrete-time sequences.
everywhere else:
Continuous-time derivation of convolution
The impulse signal will play a very important role in what follows.
Pulse and impulse signals. The unit impulse signal, written , is one at , and zero
OneThe
veryimpulse
everywhere useful way towill
else:signal think ofathe
play veryimpulse
important signal roleisinaswhat a limitingfollows. case of the pulse signal,
:
One very useful way to think of the impulse signal is as a limiting case of the pulse signal,
: Im p u lsIm esp u ls esIm p u ls es
Im p u ls e Im p u ls e Im p u ls e ResIm p uslse e Res p on s e
p on
The impulse signal will play a very important
Im p u ls e
roleImin what follows.
p u ls e Res p on s e

The impulse signal is equal to the pulse signal when the pulse gets infinitely short:
One very useful way to think of the impulse signal is as a limiting case of the pulse signal,
The
: impulse signal is equal to the pulse signal when
ImEpauchlsImesp u lsE ae ch Imtthe
Cr ea
pulse
pesu lsa e Cr ea t es a gets infinitely short:
ES a ch Im pa un dlsSeSca
ca led hCrled
ea
ift edtaes
n dapSuhlsifte ed
Im ResIm p uslse e Res p on s e
p on
ledp ua nlsde S h ift ed Im p uIm
S caIm ls ep uRes
ls epRes
on speon s e

The impulse signal is equal to the pulse signal when For

For
For exa m p le exa m p le
the pulse gets infinitely short:
a chm 2
Eexa pIm
lep u ls e Cr ea t es a
S ca led a n d S h ift ed Im p u ls e Res p on s e

Th e s u m ofTh
2
a lle tshuemimof p uals
ll et hr eesim p us ls
p on ese r es p on s es
Th e s u m of a is finpa ul is
ll tthhee im et htrem
s ys
ls e fin
es a lspses
p ron
es ys tsem
on e r es p on s e
is t h e fin a l s ys t em r es p on s e
For
exa m p le

2
Figure 2:Figure
Th e s u2: Characterizing
Characterizing
m of a ll t h e a
imlinear ap on
linear
system
p u ls e r es system
s es using itsusing its impulse
impulse response.response.
is t h e fin a l s ys t em r es p on s e
Figure 2: Characterizing a linear system using its impulse response.

The wayThe we way we impulse

use the use the impulseresponseresponse
functionfunction is illustrated
is illustrated in Fig. 2.inWeFig.conceive
2. We conceive of the input
of the input
= + + + +
... ... Thestimulus,
stimulus,
way we
in this
usethis
in thecase
stimulus,
casewould
responses
impulse
in this
weget
response
case
a sinusoid,
a sinusoid,
would
as function
a sinusoid,
asgetifimpulse
as ifisthe
if it were
ifiteach
wereimpulse
itillustrated
were
the presented
sumwas
sumthe
of presented
a set
inset
of asum Fig.of2.
of
of impulses
Weofconceive
aimpulses
set
separately (Eq.
impulses
(Eq. 1).
1).and
(i.e., We
of(Eq.
scaled
the
We input
know
1).
know Wethe
the of
and copies
shifted
know the
copies of
responses we if each was separately (i.e., scaled shifted
= + + + +
... ... responses we would
the impulse get if2:response).
the impulse
response).
Figure
each Weimpulse
simply
Characterizing Wewas simply
add presented
add
together
a linear systemallseparately
together
of theall
using of (i.e.,
(scaled
its
scaled
the and
impulse (scaled and shifted
shifted)
response. impulsecopies
shifted) impulseof responses
responses to to
the impulse
predictresponse).
predict
how thehow Wethe
system simply
system
will addwill
respond together
to theall
respond ofthe
to thecomplete
complete (scaled
stimulus. and shifted) impulse responses to
stimulus.
=
Figure 1: +
Staircase approximation+to a continuous-time signal.
+ +
... ... predict how the system will respond to the complete stimulus.
The wayNow wewe Now
usewill we
therepeat willall
impulse repeat
this inallmathematical
response this in mathematical
function isnotation. notation.
illustrated Our goalOur
in Fig. goal
is to
2. Weshowis to
that
conceiveshow ofthat
the the the
response
inputresponse
(e.g., (e.g.,
Figure 1: Staircase approximation to a continuous-time signal. Now we
membrane
stimulus, membrane
in will
this repeat a all
potential
case potential
this
fluctuation)
sinusoid, asfluctuation)
in mathematical
if itofwere of asum
shift-invariant
notation.
a shift-invariant
the ofOur goal
linear
a set linear
system
of system
isimpulses
to show
(e.g.,that(e.g.,
the
1). response
passive
(Eq. passive
neural
We know neural
the membrane)
(e.g.,
membrane)
Representing signals with impulses. Any signal can be expressed as a sum of scaled and membrane
can we
responses be can be get
potential
written
would written
fluctuation)
as aifsum
each asofaimpulse
sum
of
scaled ofandscaled
a shift-invariant
was and copies
shifted
presentedshifted
linear ofcopies
system of
the systems
separately thescaled
(e.g.,
(i.e., systems
passive
impulse
and impulse
neural
response
shifted response
membrane) of function.
function.
copies
Figure 1: Staircase approximation to a continuous-time signal.
hifted unit impulses. We
Representing Staircase approximation to continuous-
beginwith
signals with impulses.
the pulse or Any
staircase
signal can approximation
be expressed as to a continuous
a sum of scaled and
can be written
the impulse as a sumWe
response). of scaled
simplyand addshifted
together
predict how the system will respond to the complete stimulus.
copies
all ofofthe
Convolution
the(scaled
systems andimpulse
shifted)response function. to
impulse responses

ignalshifted
, as illustrated
impulses.in We
Fig.begin
1. Conceptually,
with the pulsethis is trivial: for each discrete sample of the The convolution
The convolution integral.integral. Begin by Begin
usingbyEq. using
1 toEq. 1 to replace
replace the inputthe input signal
signal by its repre-
by its repre-
riginal
unit
Representing
signal, we
signal
signals
, asmake
with
a pulse
illustrated
time signal
impulses.
in signal.
Fig. 1. Then
Any
or staircase
signal
we add up
Conceptually,
can be
thisallisthese
approximation
expressed
pulse
trivial:
as
for signals
a sum
to a continuous
of
to make
each discrete
scaled
up theofand
sample the
Now
Thesentationsentation
we will inrepeat
convolution
membraneinpotential
sentation
terms inofterms
Representing
all this
integral.
terms of fluctuation)
inof
impulses: impulses:
thebyinput
mathematical
Begin using signal
notation.
Eq. 1 toOur asgoal
a sum
replace of pulses:
is toinput
the
impulses: of a shift-invariant linear system (e.g., passive neural membrane)
show that the response
signal (e.g.,
by its repre-
shifted unitsignal.
pproximate impulses. We of begin with the pulsecanor staircase
turnupbeallapproximation to a continuous
original signal, Each
we make these pulse
a pulse signals
signal. Then we in add represented
these pulseassignals
a standard
to make pulse
up the can be written as a sum of scaled and shifted copies of the systems impulse response function.
signal
caled , as illustrated
by the appropriate in Fig.shifted
1. Conceptually, this is place.
trivial:Infor each discrete sample of the
approximate signal. value
Eachand to thesignals
of these pulse appropriate
can in turn be mathematical
represented asnotation:
a standard pulse
original signal, we make a pulse signal. Then we add up all these pulse signals to make up the past present future
scaled by the appropriate value = and
+ shifted to the appropriate
+ place.+ In mathematical + notation: The convolution 0integral.
0 0 1 Begin0 0 by0using
0 0Eq. 1input
to replace
(impulse)the input signal by its repre-
approximate signal. Each of these ... pulse signals can in turn be represented as a standard ... pulse sentation in terms of impulses:
1/8 1/4 1/2 weights
scaled by the appropriate value and shifted to the appropriate place. In mathematical notation: Using
Using additivity, additivity,
Using additivity,
0 0 0 1/2 1/4 1/8 0 0 0 output (impulse response)
Figure 1: Staircase approximation to a continuous-time signal. Using additivity,
As we let approach zero, the approximation becomes better and better, and the in the limit
Representing signals with impulses. Any signal can be expressed as a sum of scaled and Taking the limit,the
Taking 0 0limit,
0 1 1 1 1 1 1 input (step)
qualsAs we. Therefore,
let approach zero, theWeapproximation becomesapproximation
better and better, to aand the in the limit
shifted unit impulses. begin with the pulse or staircase continuous Taking the limit, Taking the1/8limit,
1/4 1/2 weights
equals . Therefore,
signal , as illustrated in Fig. 1. Conceptually, this is trivial: for each discrete sample of the
As we let approach zero,we
original signal, themake
approximation
a pulse signal. Then we becomes
add up allbetter and signals
these pulse better,toandmakethe in the limit
up the Using additivity, 0 0 0 1/2 3/4 7/8 7/8 7/8 7/8 output (step response)

approximate signal. Each of these pulse signals can in turn be represented as a standard pulse 6 6
equals . Therefore, Figure 3: Convolution as a series of weighted sums.
Also, as , thescaled by the appropriate
summation approachesvalue andanshifted to theand
integral, appropriate place.approaches
the pulse In mathematical thenotation:
unit impulse: Using homogeneity (scalar rule),
6
Taking the limit,
Also, as , the summation approaches an integral, and the pulse approaches the unit impulse: Using homogeneity,
(1)
Also, as , the summation approaches an integral, and the pulse approaches the unit impulse: (1)
As we let approach zero, the approximation becomes better and better, and the in the limit Now let Defining
be the response ofh(t)toas
the the impulse
unshifted response,
6unit impulse, i.e., . Then by using
n other words, we equalscan represent any signal as an infinite sum of shifted and scaled unit impulses. A
. Therefore, shift-invariance,
igitalIncompact
other words,
disc, we for can represent
example, stores anywhole
signalcomplex
as an infinite
pieces sumof of
musicshifted and of
as lots scaled
simple impulses.(1)A
unitnumbers (4)
digital compact
epresenting very short disc, for example,
impulses, and then stores
thewhole
CD playercomplex addspieces
all the ofimpulses
music as back lots oftogether
simple numbers
one Notice what this last equation means. For any shift-invariant linear system , once we know its
Also,can
as represent
, the summation approaches an integral, and the
of pulse
In another
fter other words,
representing we
very
to recreate short impulses,
the complex any signal
and
musical thenas the
an infinite
waveform. CD player sum adds all approaches
shifted theand impulsesthe unitunit
scaled impulse:
back impulses. A
together one impulse response (that is, its response to a unit impulse), we can forget about entirely, and
just add up scaled and shifted copies of to calculate the response of to any input whatsoever.
digital
aftercompact
another to disc, for example,
recreate the complex storesmusical
whole complex
waveform. pieces of music as lots of simple (1)
numbers Thus any shift-invariant linear system is completely characterized by its impulse response .
This no doubt
representing veryseems
shortlike a lot ofand
impulses, trouble
then the to go
CDto,playerjust to get all
adds back thethe same signal
impulses that we one
back together
riginally This
after another notowith,
started doubt
recreate
In other seems
but inthe
words, welike
fact, we
complex
can arepresent
lot musical
will of trouble
very
any astoanbe
shortly
waveform.
signal go to, sum
able
infinite just
to ofto
use get and
Eq.
shifted 1back theunitsame
toscaled
perform signal
A that we
a marvelous
impulses.
The way of combining two signals specified by Eq. 4 is know as convolution. It is such a
widespread and useful formula that it has its own shorthand notation, . For any two signals and
rick. originally starteddigitalwith,
compactbutdisc,
in for
fact, we will
example, very
stores wholeshortly
complex bepieces
ableoftomusic
use as Eq.lots1oftosimple
perform
numbersa marvelous , there will be another signal obtained by convolving with ,
This no doubt
trick. representing
seems very likeshort
a lot impulses, and thentothego
of trouble CDto,player
justadds
to allget
theback
impulsesthebacksametogether one that we
signal
after another to recreate the complex musical waveform.
originally started with, but in fact, we will very shortly be able to use Eq. 1 to perform a marvelous
trick. This no doubt seems like a lot of trouble to go to, just to get back the same signal that we
Linear Systems
originally started with, but in fact, we will very shortly be able to use Eq. 1 to perform a marvelous
Linear Systems
trick. Convolution as a series of weighted sums. While superposition and convolution may sound
Shift-invariant linear
Linear, Shift-Invariant Systems
a little abstract, there is an equivalent statement that will make it concrete: a system is a shift-
A system
Linearor transform maps an input signal into an output signal : systems and impulses
invariant, linear system if and only if the responses are a weighted sum of the inputs. Figure 3
A systemSystems
or transform maps an input signal
Linear Systems into an output signal : shows an example: the output at each point in time is computed simply as a weighted sum of the
inputs at recently past times. The choice of weighting function determines the behavior of the
system. Not surprisingly, the weighting function is very closely related to the impulse response of
A system
Linearity:
A system or transform or transform
maps an inputScalar
maps an rule
input signal
signal and
intointo additivity
ananoutput
output signal
signal : : the system. In particular, the impulse response and the weighting function are time-reversed copies
of one another, as demonstrated in the top part of the figure.
where denotes the transform, a function from input signals to output signals.
Applied to impulse, sums of impulses
where denotes the transform, a function from input signals to output signals.
Systems come where
in a wide variety
denotes of types.
the transform, One important
a function classtoisoutput
from input signals known as linear systems. To
signals.
eewhere the
Systems come
whether denotes
a system Applied
is linear,
transform, to
in a wide variety
weinneed
a sine
tovariety
function waves,
of types.
test from
whether input sums
One important
it obeys
signals
class
certain
to of is sine
known as
rules
output that
signals. waves
all
as linear
linear systems. To
linear .systems
Systems come a wide of types. One important
see whether a system is linear, we need to test whether it obeys certain rules class is known thatsystems To systems
all linear
bey. The two basicseetests
whetherof alinearity are homogeneity
system is linear, and additivity.
we need to test whether it obeys certain rules that all linear systems
obey. The two
Systems come basic
obey.inThe tests
a wide of linearity
variety
two basic tests ofof are
types.
linearity homogeneity
areOne and
important
homogeneity additivity.
class is known as linear systems. To
and additivity.
7
see whether a system is linear, we need to test4 whether it obeys certain rules that all linear systems
4
4 and additivity.
obey. The two basic tests of linearity are homogeneity

4
 Example Bass/Treble filters Convolution and multiplication
Signal Frequency content

Miles Davis time sample

Fourier transform
Half Nelson
Convolution: frequency sample
Filter
x1[n] * x2[n] X1[k] X2[k]
Bass only:
Low-pass or
Bass Filter
Multiplication: convolution

x1[n] x2[n] (1/N) X1[k] * X2[k]

Treble only:
High-pass or
Treble Filter

Time Prev
Next

The Big Payoff The Big Payoff

input signal input transform

!
The signal: f ! F F

The system T has impulse response i ! I

The system's response is T(f ) = i " f ! IF cross-correlated with convolved with ! multiplied by

! !
!
F
In other words, the Fourier transform of the impulse
response is the modulation transfer function (MTF). receptive field impulse response filter MTF

= =
The corresponding receptive field is the time- or

!
space-reversed impulse response. F

output signal output transform

 1-D Example: Gaussian Blur 1-D Example: Bandpass (DOG) Filter

! !
F F
input signal input signal

! !
F F
impulse response impulse response

! !
F F
output signal output signal

1-D Example: Bandpass (Gabor) Filter

!
F
? input signal

!
F
? impulse response

!
? F
output signal

Gabor function

1-d Example: fMRI Block alternation with Fourier transform of response with noise
noise & drift and drift
Relative neural

1
1
Relative amplitude

0.8
activity

of neural activity

0.6
HIRF 0.8

0.6 HIRF
0.4

0.2 0.4

0 0.2
0 10 20 30 40 50 60 70 80 90 100
0
(% change image intensity)

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
1
fMRI response

0.5
+ noise + drift 0.4
+ noise + drift
fMRI response
amplitude

0.3
0

0.2
-0.5

0.1
-1
0 10 20 30 40 50 60 70 80 90 100
0
Time (sec) 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5

Frequency (Hz)
Bandpass  2-D Example: Gaussian Blur

fMRI response
filtering 

!

to remove  F
input signal
noise and 
        

drift 
Time (sec)

fMRI response
amplitude


!


How do you make a  F

impulse response
low-pass filter? 
          
Frequency (Hz)
How do you make a 
high-pass filter?
fMRI response

!
F

output signal
        
Time (sec)

2-D Example: Bandpass (DOG) Filter 2-D Example: Bandpass (DOG) Filter

! !
F F
input signal input signal

! !
F F
impulse response impulse response

! !
F F
output signal output signal

2-D Example: Gabor Filter Applications: Line Spread Function

(Campbell & Gubisch, 1966)

!
F
input signal

!
F
impulse response

!
F
output signal
Applications: Line Spread Function Point-Spread Function
(Campbell & Gubisch, 1966)

Applications: Multiple Mechanisms Applications: Multiple Mechanisms

(Campbell & Robson, 1968) (Campbell & Robson, 1968)
Square wave

Near Contrast Sensitivity

Function (CSF)
Sine wave

Far

Ratio and peak-

detector model

Applications: Multiple Mechanisms Applications: Summation Within and Between

(Campbell & Robson, 1968) Channels (Graham & Nachmias, 1971)
f

CSF

3f

CSF shifted
3x left & down

Square vs sine Peaks

discrimination Subtract

Peaks
Add

Single-channel Multiple-channel
Stimulus prediction prediction
Applications: Summation Within and Between Applications: Summation Within and Between
Channels (Graham & Nachmias, 1971) Channels (Graham & Nachmias, 1971)

Applications: SF Adaptation (Blakemore & Applications: SF Adaptation (Blakemore &

Campbell, 1969) Campbell, 1969)

Applications: SF Adaptation (Blakemore & Applications: SF Adaptation (Blakemore &

Sutton, 1969) Sutton, 1969)
 Applications: Pattern Masking Applications: Pattern Masking
(Wilson et al., 1983) (Wilson et al., 1983)
Mask one pattern (Gabor, D6, ...) by another (e.g., a
sine wave grating, tilted obliquely)

Threshold is raised by the masker if channel being

used is sensitive to both

Many possible explanations of the rise in threshold

with masker contrast:

- Webers Law, possibly as a result of multiplicative

noise (noise whose SD is proportional to mean
response): !I
=k
I
- Nonlinearity followed by additive noise

Applications: Pattern Masking Applications: Pattern Masking

(Wilson et al., 1983) (Wilson et al., 1983)

Applications: Pattern Masking Applications:Detection vs. Identification,

(Wilson et al., 1983) Labeled Lines (Watson & Robson, 1981)
Applications:Detection vs. Identification, Applications:Detection vs. Identification,
Labeled Lines (Watson & Robson, 1981) Labeled Lines (Watson & Robson, 1981)

Applications:Detection vs. Identification, Applications:Detection vs. Identification,

Most Discriminating Mechanism Most Discriminating Mechanism
(Regan & Beverley, 1985) (Regan & Beverley, 1985)

Applications:Sampling and Reconstruction Applications:Sampling and Reconstruction

1.2 1

1 20
0.8 Band-limited
0.8
0.6 15 signal
0.6
0.4 10
0.4

0.2 0.2 5

0
0 10 20 30 40 50 60 0 0
20 0 20 20 0 20
Time
1 6
15 Supra-Nyquist Sufficient
0.8 5
sampling sampling
10 4
0.6
3
5 0.4
2
0.2 1
0
0 10 20 30 40 50 60
0 0
Frequency 0 20 40 60 20 0 20

Time Frequency
Applications:Sampling and Reconstruction Applications:Sampling and Reconstruction
1 8
1 1
0.8
6 0.5 0.5
0.6
4 0 0
0.4

2 0.5 0.5
0.2

0 0 1 1
20 0 20 20 0 20

sampled function
1 2
Supra-Nyquist Insufficient 1 1
0.8
sampling 1.5 sampling
0.6 0.5 0.5
1
0.4 0 0
0.5
0.2
0.5 0.5
0 0
0 20 40 60 20 0 20 1 1
Time Frequency 0 20 40 60 0 20 40 60

Applications:Sampling and Reconstruction Applications:Sampling and Reconstruction

1.2

20
Reconstruction of
1
15 sufficiently band-limited
signal
10
0.8
5
Reconstruction of
0 0.6
30 20 10 0 10 20 30 insufficiently band-limited
signal
0.4
6

5
0.2
4

3
0
2

1
0.2
0 30 20 10 0 10 20 30
30 20 10 0 10 20 30

Frequency Time

Applications:Sampling and Reconstruction Applications:Sampling and Reconstruction

sinc(x) = sin(x)/x Box Filter

!
F

!
F

Time Frequency
Sample spacing
Applications:Sampling and Reconstruction Applications:Sampling and Reconstruction

! !
F F

Applications:Sampling and Reconstruction Applications:Sampling and Reconstruction

(Yellott, 1983) (Yellott, 1983)

Fovea Parafovea (approx. 6 deg ecc.)

Applications:Sampling and Reconstruction

(Yellott, 1983)

Periphery (approx. 35 deg ecc.)