Visual working memory

Visual working memory

WM refers to the ability of the nervous system to actively internally maintain
information over brief intervals

An essential component of most complex behaviors

Closely linked to general intelligence

Limited in terms of
capacity
time

Visual working memory

Storing information across saccades
Transsaccadic integration
averaging of feature information extracted before and after the saccade
Visual working memory
The set size effect
the error with which objects are stored in memory increases with the number of stored
objects
the hallmark feature of human memory

( from the target color)

Models of working memory

The slot (or item limit) model of WM

Each of N visual items is stored in one of

a fixed number of independent memory
slots (K)
Normal (in memory)
When there are more items than slots,
(N > K) one or more items are not stored Uniform (not in memory)

errors in report of a randomly chosen item

will be composed of a mixture of high-
precision responses and random guesses
Capacity K = N x pt
K ~ 3-4
pt = 1
else pt = K/N

Models of working memory

Resource models of working memory

A limited supply of representational

medium that is shared out between
items, without a limit to the number of
items that can be stored
The precision with which an item can be
recalled depends on the quantity of
resource allocated to it
Error variability (width of the
distribution) increases continuously with
the number of items
Models of working memory
Slots + averaging

Hybrid models combine resource-like

behavior at lower set sizes with a fixed
upper limit on items stored
the working memory medium is divided
into a discrete number of quanta (slot models)
slots are shared out between items
(resource models)

Models of working memory

Variable precision models

Precision varies, from trial to trial and

item to item, around a mean that
decreases with increasing number of
items as a result of limited resources
Recall errors will be made up of an
infinite mixture of distributions (assumed
normal) of different widths
Variability in precision could stem from
variability in resource or from bottom-up
factors

Models of working memory

Resource models (ER & VP) are better in

reproducing human response
distributions
Flexibility in VWM resource allocation

Flexibility in VWM resource allocation

Pre-cue paradigm
one stimulus, indicated by a cue, is more likely to be selected for test
recall is consistently found to be less variable for the cued item than for uncued items
the allocation observed in human participants is consistent with an optimal strategy
aimed at minimizing total error variability in the task

Flexibility in VWM resource allocation

Manipulation of incentives
items that were marked with a pre-cue as yielding higher reward were remembered better
WM and attention
Retro-cue paradigm
Precision is better for the retro-cued item

Flexibility, but not in encoding

protection from time-based decay
removal of obsolete objects
attentional strengthening (refreshing)
protection from (perceptual) interference

Flexibility in VWM resource allocation

Stored information can quickly become outdated
Updating mechanisms are needed to ensure the contents of memory remain
relevant to current task goals

Flexibility in VWM resource allocation

VWM can be efficiently updated when previous information becomes obsolete
This is a demanding active process that occasionally fails intrusion errors
Binding and swap errors
Objects in our visual world are characterized by a combination of visual features
To faithfully memorize a scene, we need to encode not only the individual
features that are present but also their specific combinations that constitute
different objects

Binding refers to the information that groups different features together into
objects
Binding errors
common in memory
Binding errors

Binding and swap errors

Swap errors
reporting the feature value of a non-target item
their distribution with respect to the target feature is random/uniform
may be mistaken for random guesses if only the raw deviation of responses from the
target is examined

Binding and swap errors

Swap errors increase with increasing
variability in memory for the cue features
Changing frequency of swap errors can be
quantitatively predicted based on recall
variability when the same feature was used
for report

Swap errors can be wholly accounted for by

confusions due to cue-dimension variability
Measuring VWM uncertainty
Uncertainty (confidence) should
increase (decrease) with the number
of items to be remembered

Ordinal scale

Measuring VWM uncertainty

Uncertainty (confidence) should
increase (decrease) with the number
of items to be remembered

Ordinal scale

Error distributions can be

decomposed on the basis of
subjective confidence ratings into
components that differ in precision

Measuring VWM uncertainty

Uncertainty (confidence) should
increase (decrease) with the number
of items to be remembered

Confidence interval

Arc length is correlated with absolute

error
Measuring VWM uncertainty
Uncertainty is maintained in WM, and can be estimated continuously not just
whether the memory is present or absent.
At a fundamental level, this means that WM is much richer than previously
believed.
Metacognition
reflective monitoring and control processes
cognition about cognition

VWM in a structured environment

The information we need to hold in WM in real world situations is generally statistically
structured and predictable
We have prior knowledge that can help constrain our memories
To store information successfully in WM, we need to communicate to our future selves only
what is unexpected or unknown about the given object or scene

Information theory
compression and rate-distortion theory
attempt to formalize the entropy of the stimulus and the communication problem faced by our memory
system
Efficient coding theory
encoding resources are preferentially allocated to more frequently encountered stimuli in order to maximize
the information transmitted
e.g., cardinal orientations are more prevalent than obliques in natural scene

Changes in VWM over delay

Rapid deterioration over seconds is a defining feature of VWM
This effect is relatively smaller in comparison to the strong and robust effects of set size
Changes in VWM over delay
A feature value encoded in a population code

on the stored value

Decay
diffusion
neural activity representing objects will decrease
over time
Diffusion
active regions will diffuse randomly over time due to
noise in neural activity (i.e., random walk)
Interference
objects in memory compete with each other
biases, fusion

All accounts predict increasing recall variability

with time

Neural mechanisms of VWM

Population coding
Normalization seems to be a common operation in neural systems
rescaling a set of values to have a particular common magnitude
total spiking activity is limited due to normalization, and is distributed (shared out)
between visual items
provides a plausible biological basis for the concept of a limited WM resource

Neural mechanisms of VWM

Population coding
Normalization
Neural correlates of storage in VWM
Inverted encoding models
reconstructing population-level stimulus representations from aggregate measurements
of neural activity (e.g., fMRI or EEG)

Neural correlates of storage in VWM

Neural correlates of storage in VWM

Neural sites of VWM:
Occipital cortex (OC)
sensory recruitment hypothesis
early sensory (occipital) areas are involved in the representation of visual memories
Prefrontal cortex (PFC)
Posterior parietal cortex (PPC)

Neural mechanisms of VWM storage:

Activity silent coding
temporary changes in synaptic efficacy (similar to LTM)
Persistent activity
neural activity that takes place after a sensory stimulus is no longer present and that holds
information about that stimulus
Neural correlates of storage in VWM
Features in sensory regions versus prefrontal cortex differ in the level of abstractness and
generalizability
Features in prefrontal cortex reflect representations that are transformed for guidance of
upcoming behavioral actions
Distributed representations
neural representations of a memorized content (i.e., stimulus information) can be found in multiple neural
populations