The Journal of Neuroscience, October 13, 2021 • 41(41):8577–8588 • 8577

Systems/Circuits

Identification of Pattern Completion Neurons in Neuronal

Ensembles Using Probabilistic Graphical Models
Luis Carrillo-Reid,1p Shuting Han,1p Darik O’Neil,1p Ekaterina Taralova,1,2 Tony Jebara,2 and Rafael Yuste1
1 2
Departments of Biological Sciences and, and Computer Science, Columbia University, New York, New York 10027

Neuronal ensembles are groups of neurons with coordinated activity that could represent sensory, motor, or cognitive states.
The study of how neuronal ensembles are built, recalled, and involved in the guiding of complex behaviors has been limited
by the lack of experimental and analytical tools to reliably identify and manipulate neurons that have the ability to activate
entire ensembles. Such pattern completion neurons have also been proposed as key elements of artificial and biological neural
networks. Indeed, the relevance of pattern completion neurons is highlighted by growing evidence that targeting them can
activate neuronal ensembles and trigger behavior. As a method to reliably detect pattern completion neurons, we use condi-
tional random fields (CRFs), a type of probabilistic graphical model. We apply CRFs to identify pattern completion neurons
in ensembles in experiments using in vivo two-photon calcium imaging from primary visual cortex of male mice and confirm
the CRFs predictions with two-photon optogenetics. To test the broader applicability of CRFs we also analyze publicly avail-
able calcium imaging data (Allen Institute Brain Observatory dataset) and demonstrate that CRFs can reliably identify neu-
rons that predict specific features of visual stimuli. Finally, to explore the scalability of CRFs we apply them to in silico
network simulations and show that CRFs-identified pattern completion neurons have increased functional connectivity. These
results demonstrate the potential of CRFs to characterize and selectively manipulate neural circuits.
Key words: Conditional random fields; graph theory; neuronal ensembles; pattern completion; probabilistic graphical
models; two-photon optogenetics

Significance Statement
We describe a graph theory method to identify and optically manipulate neurons with pattern completion capability in mouse
cortical circuits. Using calcium imaging and two-photon optogenetics in vivo we confirm that key neurons identified by this
method can recall entire neuronal ensembles. This method could be broadly applied to manipulate neuronal ensemble activity
to trigger behavior or for therapeutic applications in brain prostheses.

Received Jan. 9, 2021; revised July 6, 2021; accepted July 11, 2021.
Author contributions: L.C.-R., S.H., E.T., and R.Y. designed research; L.C.-R. and S.H. performed research;
L.C.-R., S.H., D.O., E.T., and T.J. contributed unpublished reagents/analytic tools; R.Y. assembled and directed
the team and secured funding and resources; L.C.-R., S.H., and D.O. analyzed data; L.C.-R. and R.Y. wrote the
paper.
This work was supported by grants from the National Eye Institute (R01EY011787), National Institute of
Neurological Disorders and Stroke (R01NS110422 and R34NS116740), National Institute of Mental Health
(R01MH115900), National Science Foundation (CRCNS 1822550), Vannevar Bush Faculty Fellowship (ONR
N000142012828) to R.Y., Consejo Nacional de Ciencia y Tecnología (CF6653 and CF154039) and Programa de
Apoyo a Proyectos de Investigación e Innovación Tecnológica (IA201421) to L.C-R, Howard Hughes Medical
Institute International Student Research Fellowship (S.H.), and National Science Foundation Graduate Research
Fellowship (D.O.). We thank Kui Tang for code; laboratory members for comments and virus injections;
Columbia University Yeti and Habanero shared High Performance Computing Cluster for computation
resources; and Stanford University Neuroscience Gene Vector and Virus Core for the AVVdj virus.
*L.C.-R., S.H., and D.O. contributed equally to this work.
L. Carrillo-Reid’s present address: Neurobiology Institute, National Autonomous University of Mexico,
Juriquilla, 76230 Queretaro, Mexico
The authors declare no competing financial interests.
Correspondence should be addressed to Luis Carrillo-Reid at carrillo.reid@comunidad.unam.mx.
https://doi.org/10.1523/JNEUROSCI.0051-21.2021
Copyright © 2021 the authors
Introduction
Following Lorente de Nó’s (1938) proposal that cortical circuits
are designed to sustain recurrent reverberating chains of activity,
Hebb (1949) postulated that recurrent neuronal activity could
generate cell assemblies, functional multineuronal units, as
building blocks within neuronal microcircuits. In addition, Hebb
speculated that strong recurrent connections between a given set of
neurons would increase the probability to reactivate a whole group
if only one of the elements was activated. This property, known as
pattern completion, represents an efficient mechanism to retrieve
information allowing the reactivation of previously stored informa-
tion triggered by a few neurons (Marr, 1971; Hopfield, 1982; Rolls
and Treves, 1994). Acknowledging this tradition, we use the term
“pattern completion” to define neurons whose activation can recall
an entire ensemble (Carrillo-Reid et al., 2016, 2019).
A neuronal ensemble is a group of neurons with coordinated
activity that represents an experimental or behavioral condition
(Carrillo-Reid and Yuste, 2020a,b). Advances in two-photon cal-
cium imaging and two-photon optogenetics have made possible
the simultaneous reading and writing of cortical ensemble
8578 • J. Neurosci., October 13, 2021 • 41(41):8577–8588
activity with single-cell resolution in awake animals (Rickgauer
et al., 2014; Packer et al., 2015). Coactivating sets of neurons can
artificially imprint neuronal ensembles in vivo (Carrillo-Reid
et al., 2016). Neuronal ensembles can also be recalled by tar-
geting individual neurons with two-photon holographic
optogenetics, demonstrating pattern completion and causal-
ity between neuronal ensemble activity and learned behav-
iors (Carrillo-Reid et al., 2019; Marshel et al., 2019; Robinson
et al., 2020). The identification of pattern completion neu-
rons (p.c. neurons) could enable the systematic manipulation
of neuronal ensembles.
Graph theory has been used in neuroscience to describe the
structural and functional organization of entire brains (Bullmore
and Sporns, 2009). Graphs are usually constructed with nodes
representing brain regions (Bettencourt et al., 2007) and edges
representing information flow (Iturria-Medina et al., 2008).
Many studies have constructed graphs for functional analysis of
brain data from fMRI, EEG, and electrode arrays, taking brain
regions (Achard and Bullmore, 2007; Fair et al., 2008; Hagmann
et al., 2008), voxels (Eguíluz et al., 2005; van den Heuvel et al.,
2008; Zuo et al., 2012) or electrode position (Downes et al., 2012)
as nodes, and cross-correlation, mutual information, or Granger
causality as edges (Fair et al., 2008; Bullmore and Sporns, 2009;
Micheloyannis et al., 2009; Wang et al., 2010; Khazaee et al.,
2015). Graphs are usually associated with a restricted set of pa-
rameters that describe the weight and direction of edges obtained
by pairwise metrics, underscoring network interactions. To
understand the role of pattern completion neurons in awake ani-
mals it is necessary to generate graphical models that capture the
interaction between neurons and subsequently identify and
manipulate individually neurons with pattern completion capa-
bility that could have a potential role orchestrating network ac-
tivity (Carrillo-Reid and Yuste, 2020a).
Neuronal ensembles form a network structure that can be
naturally characterized with probabilistic graphical models,
where nodes and edges are biologically meaningful, representing
neurons and their functional connections, respectively. Here, we
used conditional random fields (Koller and Friedman, 2009), a
combination of graph theory and probabilistic modeling that
uses graphs to simplify and express the conditional interaction
among a collection of variables. We applied CRFs to existing
data from a recent work that used two-photon optogenetics to
investigate pattern completion properties of neurons from
mouse primary visual cortex in vivo (Carrillo-Reid et al., 2016),
and we demonstrate that the parameters obtained from CRFs
can predict neurons with the ability to recall artificially imprinted
ensembles. To test the broad applicability of CRFs, we also ana-
lyzed publicly open calcium imaging data (Allen Institute Brain
Observatory dataset) and demonstrated that our approach can
identify neurons that reliably predict different features of visual
stimuli and could potentially be targeted to recall percepts.
Finally, to test the scalability of CRFs we used in silico network
simulations and demonstrated that few neurons from Hopfield-
style networks have pattern completion capability and increased
functional connectivity.
Materials and Methods
Animals and surgery. All experimental procedures were conducted
in accordance with the National Institutes of Health and Columbia
University Institutional Animal Care and Use Committee and have been
described previously (Carrillo-Reid et al., 2016, 2019). Briefly, simultane-
ous two-photon imaging and two-photon optogenetic experiments were
performed on C57BL/6 male mice. Virus AAV1-syn-GCaMP6s-WPRE-
Carrillo-Reid et al. · Identification of Pattern Completion Neurons
SV40 and AVVdj-CaMKIIa-C1V1(E162T)-TS-P2A-mCherry-WPRE were
injected simultaneously into layer 2/3 of left primary visual cortex (2.5 mm
lateral and 0.3 mm anterior from the l , 200 mm from pia). After 3 weeks
mice were anesthetized with isoflurane (1–2%) and a titanium head plate
was attached to the skull using dental cement. Dexamethasone sodium
phosphate (2 mg/kg) and enrofloxacin (4.47 mg/kg) were administered sub-
cutaneously. Carprofen (5 mg/kg) was administered intraperitoneally. After
surgery animals received carprofen injections for 2 d as postoperative pain
medication. A reinforced thinned skull window for chronic imaging (2 mm
in diameter) was made above the injection site using a dental drill. A 3 mm
circular glass coverslip was placed and sealed using a cyanoacrylate adhesive
(Drew et al., 2010). Imaging experiments were performed 7–28 d after head
plate fixation. During recording sessions mice were awake (head fixed) and
could move freely on a circular treadmill.
Visual stimulation. Visual stimuli were generated using MATLAB
Psychophysics Toolbox and displayed on an LCD monitor positioned
15 cm from the right eye at 45° to the long axis of the animal. Population
activity corresponding to two-photon stimulation of targeted neurons in
layer 2/3 of visual cortex was recorded with the monitor displaying a
gray screen with mean luminescence similar to drifting gratings. The
imaging setup and the objective were completely enclosed with blackout
fabric and a black electrical tape. Visual stimuli consisted of full-field
sine wave drifting gratings (100% contrast, 0.035 cycles/°, 2 cycles/s)
drifting in two orthogonal directions presented for 4 s, followed by 6 s of
mean luminescence (Carrillo-Reid et al., 2016, 2019).
Simultaneous two-photon calcium imaging and photostimulation.
Two-photon imaging and optogenetic photostimulation were per-
formed with two different femtosecond-pulsed lasers attached to a
commercial microscope. An imaging laser (l = 940 nm) was used to
excite a genetically encoded calcium indicator (GCaMP6s), and a
photostimulation laser (l = 1064 nm) was used to excite a red
shifted opsin (C1V1), which preferentially responds to longer wave-
lengths (Packer et al., 2012).
The two laser beams on the sample were individually controlled by
two independent sets of galvanometric scanning mirrors. The imaged
field of view was ;240
240 mm (25
numerical aperture 1.05, XLPlan
N objective), comprising 60–100 neurons.
We adjusted the power and duration of photostimulation so that
the amplitude of calcium transients evoked by C1V1 activation
mimic the amplitude of calcium transients evoked by visual stimula-
tion with drifting gratings. Single-cell photostimulation was per-
formed with a spiral pattern delivered from the center of the cell to
the boundaries of the soma as has been previously published
(Carrillo-Reid et al., 2016).
Image processing. Image processing was performed with ImageJ (ver-
sion 1.42q, National Institutes of Health) and custom-made programs
written in MATLAB as previously described (Carrillo-Reid et al., 2016,
2019). Acquired images were processed to correct motion artifacts using
TurboReg. Neuronal contours were automatically identified using inde-
pendent component analysis and image segmentation (Mukamel et al.,
2009). Calcium transients were computed as changes in fluorescence:
(Fi–Fo)/Fo, where Fi denotes the fluorescence intensity at any frame, and
Fo denotes the basal fluorescence of each neuron. Spikes were inferred
from the first-time derivative of calcium signals with a threshold of 3
SDs above noise level. We constructed an N
F binary matrix, where N
denotes the number of active neurons, and F represents the total number
of frames for each movie.
Population vectors. We constructed multidimensional population
vectors that represent the simultaneous activation of different neurons.
Peaks of synchronous activity describe population vectors (Carrillo-Reid
et al., 2008). Only high-activity frames were used. An activity threshold
was determined by generating 1000 shuffled raster plots and comparing
the distribution of the random peaks against the peaks of synchrony
observed in the real data (Shmiel et al., 2006). We tested the significance
of population vectors against the null hypothesis that the synchronous
firing of neuronal pools is given by a random process (Shmiel et al.,
2006; Carrillo-Reid et al., 2015a). Such population vectors can be used to
describe the network activity as a function of time (Schreiber et al., 2003;
Stopfer et al., 2003; Brown et al., 2005; Sasaki et al., 2007; Carrillo-Reid
Carrillo-Reid et al. · Identification of Pattern Completion Neurons J. Neurosci., October 13, 2021 • 41(41):8577–8588 • 8579

et al., 2008). The number of dimensions for each experiment is given by imaging data, we constructed CRF models in the following two steps: (1)
the total number of active cells (N). Neuronal ensembles are defined by structure learning and (2) parameter learning.
the concomitant firing of neuronal groups at different times. The simi- For structure learning, we learned a sparse graph structure
larity between two population vectors was defined by their normalized G ¼ ðV; AÞ using ‘1 -regularized neighborhood-based logistic regression
inner product, which represents the cosine of the angle between such for each node r (Ravikumar et al., 2010) as follows:
vectors. A similarity index close to 1 indicates that both vectors point to
a close direction in an N-dimensional space, therefore the neurons that n o
define each vector are almost the same. A similarity index of 0 denotes min ‘ðu s ; xÞ1l s ku snr k1 ;
u nr
that the pair of vectors are orthogonal, so the neurons that define each
vector are different (Carrillo-Reid et al., 2015a,b; Carrillo-Reid and
where
Yuste, 2020b).
Allen Institute Brain Observatory dataset. To demonstrate the gen-  X 
eral applicability of our approach we analyzed a publicly available dataset
1
X
n exp 2xr u srt xt
 X
t2Vnr
from the Allen Brain Observatory (http://observatory.brain-map.org/ ‘ ð u s ; xÞ ¼  log 
visualcoding) along with the software development kit (SDK) for ex- n
i¼1 exp 2xr u srt xt 11
tracting fluorescence and (Fi–Fo)/Fo (http://alleninstitute.github.io/ t2Vnr

AllenSDK/) by the Allen Institute of Brain Science (de Vries et al., 2020).
Spikes were detected by first low-pass filtering the (Fi–Fo)/Fo traces,
then a threshold of 5 SDs above noise level on the first derivative of
filtered (Fi–Fo)/Fo. The experiments identifiers s used are 511507650,
511509529, 511510650, 511510670, 511510718, and 511510855.
Conditional random fields. We constructed CRFs as previously
published

(Tang  et al., 2016), using indicator feature vectors
x ¼ x1 ; x2 ; :::; xM ; where xm 2 X N , for
each edge and
 node, and target
binary population activity vectors y ¼ y1 ; y2 ; :::; yM ; where ym 2 Y N
for N neurons and M samples (time points). The X N and Y N describe
an N-dimensional space where the dimensionality describes the total
number of active neurons. For each sample, the conditional probability
can be expressed as follows:
 
  exp h f ðxm ; ym Þ; u i
and

u snr ¼ u sru ; u 2 Vnr ;
where nr denotes except r, and u s is a vector of regression parameters
for structure learning. Here, l s is a regularization parameter that con-
trols the sparsity (or conversely, the density) of the constructed graph
structure. This is essentially a logistic regression of variable Xr on the
other variables Xnr , with ‘1 -regularization. The regression coefficients
thus represent the neighborhood structure and the sign pattern. We
implemented ‘1 regularization using a highly efficient elastic-net algo-
rithm solving using penalized maximum likelihood as follows:
X
N
 

ð1  a Þ

p ym jxm ; u ¼ ; 1
wi l yi ; b 0 1 b xi 1l s k b k2 1 a k b k1 ;
2
Z ð xm ; u Þ min T
b 0 ;b N 2
i¼1

where f is a vector of the distribution expressed in log-linear form, u is

a vector of parameters with parameters for the log-linear distribution, where a = 1, and thus is equivalent to a standard LASSO (least absolute
and Z is the partition function as follows: shrinkage and selection operator). We constructed a large number of
structures (1001) across a broad range of l s , from which we stochasti-
X   cally selected a number of structures for subsequent parameter learning.
Zðxm ; u Þ ¼ exp h f ðxm ; yÞ; u i : We ensured proper bracketing by including the largest and smallest l s
y2Y structure in this priming set. In future iterations of parameter learning,
the selected sets of structures were selected using the log likelihood of
The conditional probability can be factored over a graph structure learned models as feedback. The final graph structure was obtained by
G ¼ ðV; AÞ, where V is the collection of nodes representing observation thresholding the edge potentials with a given density preference d. Edges
variables and target variables, and A is the collection of subsets of V. with potential values within the top d quantile were kept as the final
Based on the graph structure, the model parameters can be written sepa- structure. It is worth noting that although d could bias the result, varying
rately for nodes and edges as f ¼ f f V ; f A g and u ¼ fu V ; u A g 2 <N . d does not lead to density values that differ much. This is because of the
Given binary x and y, node parameters f V and u V include two sets of sparsity induced by the ‘1 regularizer.
distribution and the parameters corresponding to the node state 0 and 1, For parameter learning, we aim to learn the edge and node potential
whereas edge parameters f A and u A include four sets of distribution parameters ¼ fu V ; u A g. Based on the learned structure, we used the
and the parameters, corresponding to the edge states 00, 01, 10, and 11 Bethe approximation to approximate the partition function and iterative
(see Fig. 2). The conditional dependencies can be then written as Frank–Wolfe methods to perform parameter estimation by maximizing
follows: the log likelihood of the observations with a quadratic regularizer (Tang
et al., 2016) as follows:
X X 
u i f i ðX; Yi Þ 1 u a f aðX; YaÞ
pðYjX; u Þ ¼
exp
i2V a2A
:
X
M
lp
ZðX; u Þ ‘ðu ; X; Y Þ ¼ ‘ð u ; X m ; Y m Þ  ku k :
2
2
m¼1

This model is a generalized version of Ising models, which have been

previously applied to model neuronal networks (Schaub and Schultz,
2012). The log likelihood of each observation can be then written as
follows:
‘ðu ; Xm ; Y m Þ ¼ h f ðXm ; Y m Þ; u i  logZðXm Þ:
To reduce the complexity of the model, we first learn a sparse graph
structure that represents variable dependencies, then we construct a CRF
on the learned structure. Given the inferred binary spikes from raw
Here, l p is a regularization that controls the learned parameters and
helps prevent overfitting. Cross-validation was done to find the best l s ,
d, and l p via held-out model likelihood. We varied l s with six values
between 0.002 and 0.5, d with six values between 0.25 and 0.3, and l p
with five values between 10 and 10 000, all sampled uniformly. To obtain
the best model parameters, 90% data were used for training, and 10%
data were withheld. Cross-validation was done using all possible combi-
nations of the above parameters and calculating the likelihood of the
withheld data. Then, the best model parameters were determined by
selecting the parameter set with a locally maximum likelihood in the
8580 • J. Neurosci., October 13, 2021 • 41(41):8577–8588 Carrillo-Reid et al. · Identification of Pattern Completion Neurons

A population vectors (binary representation)

B population vectors (multidimensional representation)

1 0 1 0 0 1 0 0 0 1
1 1 1 1 1 1 1 1 1 1 d2
0 0 0 0 1 0 1 0 1 0
vectors dimensionality

0 1 0 1 0 0 0 1 0 0
1 0 1 0 0 1 0 0 0 1 population vector
0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 1 0 1 0
neurons

0
1
0
0
1
1
0
1
0
0
1
1
0
1
0
0
1
0
... 0
1
0
0
1
1
0
1
0
0
1
0
v1
v2 neuronal ensemble
1 0 1 0 0 1 0 0 0 1
0 0 0 0 0 0 0 0 0 0 cluster of vectors
0 0 0 0 1 0 1 0 1 0
0 0 0 0 0 0 0 0 0 0
0 1 0 1 0 0 0 1 0 0
1 0 1 0 0 1 0 0 0 1 d1
0 0 0 0 1 0 1 0 1 0
0 1 0 1 0 0 0 1 0 0
v1 v2 ... recurrent population vector ... vt
dn N dimensions (neurons)

time

Figure 1. Characterization of neuronal ensembles as multidimensional vectors. A, Population activity represented as population vectors. The activity from the entire popu-
lation at each time point (t) is represented by a binary vector (v1 to vt). B, Schematic representation of population vectors in an N-dimensional space, where n denotes the
total number of observed neurons. Each point in a multidimensional space represents a population vector defined by a coactive group of neurons (left). Dashed arrows indi-
cate the dimensionality of the data (d1 to dn). For clarity only population vectors v1 and v2 are depicted by solid arrows. A neuronal ensemble could be defined by a cluster
of population vectors (right).

parameter space. This empirically results in

models with connectivity density of ;10 to
40%, corresponding to the previously reported
connectivity probability in layer 2/3 of V1 A in vivo two-photon B conditional random fields
(Yoshimura et al., 2005; Ko et al., 2011).
Node strength. As 0 and 1 on graph nodes x2 x3
photostim
imaging

represent the nonactive and active state of xsn

visual stim
x1
neurons, the edge potential term 11 represents x4
xs1 x6 x5
the coactive state of connected neurons,
whereas 01 and 10 represent antiactivate, and y2 y3
00 represent the state that both nodes (neu- ysn
y1
rons) are nonactive. Because the nodes are y4 0 ф
0
mostly inactive during recording, the 00 term head fixed
y5 0 1 1 ф1
is the strongest term, whereas only the 11 free movement ys1 0 ф00 ф01
y6
term captures neuronal ensemble coactiva- 0 ф0 1 ф ф
10 11
tion. Therefore, we defined the node strength 1 ф1
as the sum of the 11 term of edge potentials
from all connecting edges as follows:
C model 1 D stimuli classification from CRFs E
y2
X
log(ps1)-log(ps2)

N ðiÞ
E ****
y1 y3 100
sðiÞ ¼ f 11 ði; jÞ:
ys1 ysn 80
accuracy (%)

j¼1
y4
y6 60
Here, NE ðiÞ denotes the number of con- y5
necting edges for node i. The defined node 40
strength reflects the importance of a given cell model n
log(ps2)-log(ps1)

y2 20
in coactivating other cells. y3
y1
Shuffling method. To generate shuffled 0
el

models, we first randomized the spike raster ys1 ysn

od
el
od

matrices while preserving the activity per cell

m

y4
fle
al

y6
uf
re

and per frame. Then, we trained CRF models

sh

using the shuffled spike matrices, with the y5

cross-validated l s , d, and l p from the real

model. This procedure was repeated 100
times. The random level of node strengths
was determined by averaging node strengths
from shuffled models.
Identifying putative pattern completion
neurons. To find the most influential neurons
for each condition, we iterated through all the
neurons and identified their contribution in
predicting the stimulus conditions in the pop-
ulation. To this end, for the ith neuron in the
Figure 2. Classification of visual stimuli from calcium imaging data using CRFs. A, In the experimental setup, simultaneous
two-photon imaging and two-photon optogenetics were performed in layer 2/3 of primary visual cortex in head-fixed freely
moving mice. B, Graphical representation of CRFs. Circles represent neurons, squares represent added nodes depicting visual
stimuli, shaded nodes (x) represent observed data, and white nodes (y) represent true states of the neurons and are connected
by edges that indicate their mutual dependencies. Node potentials are defined over the two possible states of each node, and
edge potentials are defined over the four possible states of each existing edge, depending on the state of adjacent nodes. C,
Representation of graphical structure with added stimulus nodes. Each stimulus introduces an additional node to the graph. D,
Ratio of log likelihood predicting horizontal (top, red) and vertical (bottom, blue) drifting gratings calculated by CRF models.
Colored stripes indicate visual stimuli. Scale bar, 10 s. E, The accuracy of real models is higher compared with shuffled models
(n = 8 mice, 16 models; p , 0.0001 (****); Mann–Whitney test).
Carrillo-Reid et al. · Identification of Pattern Completion Neurons J. Neurosci., October 13, 2021 • 41(41):8577–8588 • 8581

A B

log(pi,1)-log(pi,0)
first node ith node last node

4 4
4

... ...

log(pi,1)-log(pi,0)
C D E
1
1 1

p.c. neurons
low
0.8
0.8 0.8
****
0.6 0.6 0.6
AUC

AUC

AUC
0.4 0.4

non-p.c. neurons
high 0.4
0.2 0.2
0.2
0 0
-1 -0.5 0 -1 -0.5 0
node strength node strength 0

Figure 3. Identification of putative pattern completion neurons using CRFs. A, Schematic representation of putative pattern completion neurons extraction from CRF models with added
nodes for two different visual stimuli. The activity of the ith neuron is set to 1 or 0 at each frame, and the log likelihood pmi;1 and pmi;0 of modified population vectors is calculated. B, Log-likeli-
hood inference and prediction for cortical putative pattern completion neurons representing horizontal (top, red) or vertical (bottom, blue) drifting gratings. C, Graphical representation of node
strength. D, Putative pattern completion neurons are identified by high AUC and high node strength (top right quadrant). Confidence levels were defined from CRF models of shuffled data
(gray bars). E, AUC of putative pattern completion neurons with high node strength is higher than nonpattern completion neurons (n = 14 mice, 28 models; p , 0.0001 (****); Mann–
Whitney test).

population, we compared its activity or inactivity in all M frames for
each model. With the two resulting population vectors in the mth frame
among all samples, we calculated the log probability of them coming
from the trained CRF model as follows:
 
pmi;1 ¼ p ym jxmni ; xmi ¼ 1; u
 
pmi;0 ¼ p ym jxmni ; xmi ¼ 0; u :
Then, we computed the log likelihood ratio as follows:
 Data analysis. CRF models were trained using the Columbia
‘i;10 ¼ log pmi;1  log pmi;0 ; m ¼ 1; :::; M
and calculated the standard receiver operating characteristic (ROC)
curve with the ground truth as the timing of each presented visual stim-
uli. The prediction ability of all nodes for all presented stimuli is then
represented by an area under curve (AUC) matrix A, where Ai;d repre-
sents the AUC value of node i predicting stimulus d. Additionally, we
calculated the node strength S ¼ fsi g of each neuron in the CRF model.
We then computed prediction AUC Ar and node strength Sr of each
node r from 100 CRF models trained on shuffled data. The final ensemble
for stimulus d is defined as follows: fijAi;s . meanðArs Þ 1 stdðArs Þ;
Si . meanðSr Þ 1 stdðSr Þg.
Neural network simulation to study pattern completion. The simu-
lated neural network was generated using 10 Hopfield nets embedded in
a larger matrix. Each Hopfield network was generated synchronously
and deterministically (Gerstner et al., 2014). The binary state S of neuron
i in the following time step (t 1 1) given j inputs and weight wij was
described by the following:
X
Si ðt11Þ ¼ wij Sj ðtÞ:
j Neuronal ensembles as multidimensional population vectors
Every neuron was fully connected according to the weight matrix w,
defined by the following equation:
1
X
wij ¼ ensui ensuj ;
N
u
where ensiu is the state of neuron i in the uth ensemble. Only one ensem-
ble was active at any given time. External stimuli were produced by initi-
alizing the network to a noisy ensemble state (20% noise), and thereafter
the network was permitted to iterate for 5 time steps to simulate the pat-
tern completion of the ensemble. In total, the matrix contained 810 neu-
rons and 100 ensembles. The probability of any given neuron being
active during an ensemble external to its respective Hopfield net was
fixed at 2.5%.
University Yeti and Habanero shared High Performance Computing
Cluster (see Figs. 2–6). Benchmarking and simulation modeling were
performed on a custom-built computer using an Intel Core i9-9900K
processor and NVIDIA Titan RTX (see Figs. 7, 8). MATLAB R2016a,
R2019b, and R2020a (MathWorks) were used for data analysis.
We didn’t use any statistical method to predetermine sample size.
Sample sizes are similar to those reported previously (Carrillo-Reid et
al., 2016, 2019). Statistical tests were done in GraphPad Prism 5.
Statistical details of each specific test can be found in the article and the
figure legends.
Data availability. Step-by-step implementation of the code and associ-
ated notations can be found at the following: https://github.com/hanshuting/
graph_ensemble for the code used to implement CRF models on a cluster
and https://github.com/darikoneil/Identification-of-Pattern-Completion-
Neurons-in-Neuronal-Ensembles-using-Probabilistic-Graphical-Mod
for the code used to run simulated data and CRF models on a single
computer.
Results
As neuroscience moves from the study of individual neurons to
neural ensembles as the substrate of brain functions, it is neces-
sary to develop methods to identify neurons that are able to recall
8582 • J. Neurosci., October 13, 2021 • 41(41):8577–8588
entire ensembles (Carrillo-Reid et al., 2017). The development of
optical techniques has allowed the simultaneous recording and
manipulation of neurons with single-cell precision (Rickgauer et
al., 2014; Packer et al., 2015). However, studies performing si-
multaneous recordings usually perform pair correlations or the
average of all recorded neurons as a measurement of network ac-
tivity. As neuronal ensembles are related to specific functions
that can be repeated at different times, they can be optimally
defined by binary multidimensional population vectors (see
above, Materials and Methods) where each vector represents the
activity of all observed neurons at a given time window, and the
dimensionality of the vectors is defined by the number of
observed neurons (Fig. 1A). In this context, neuronal ensembles
could be understood as clusters of population vectors that point
in a similar direction in a high-dimensional space (Fig. 1B). The
advantage of conceptualizing neuronal ensembles as population
vectors, instead of pairwise correlations between neurons or
averaged activity, is that large datasets could be visualized as
changes in population vectors in real time, whereas pairwise cor-
relations of neurons are dependent on the recording time length
(Carrillo-Reid and Yuste, 2020b). Thus, we built CRFs using
population vectors that take into account all the recorded neu-
rons simultaneously, allowing the comparison of network activ-
ity under different experimental conditions to highlight neurons
with pattern completion capability that could be used to orches-
trate entire functional networks.
Building CRF models from population responses to oriented
stimuli
Cortical neurons from layer 2/3 in primary visual cortex are
organized as neuronal ensembles, where each ensemble depicts a
given orientation of visual stimuli (Miller et al., 2014; Carrillo-
Reid et al., 2015b). To investigate the ability of CRF models to
identify groups of neurons that could recall the representation of
a specific orientation of visual stimuli, we analyzed population
responses to drifting gratings from layer 2/3 neurons of primary
visual cortex in awake head-fixed mice from previously pub-
lished data (Carrillo-Reid et al., 2016; Fig. 2A).
CRFs model the conditional distribution p(y|x) of a network,
where x represents observations and y represents labels associ-
ated with a graphical structure (Sutton and McCallum, 2012). As
no assumptions are made on x, CRFs can describe the condi-
tional distribution with complex dependencies in observation
variables associated with a graphical structure that is used to
constrain the interdependencies between labels (Fig. 2B).
CRFs have been successfully applied in diverse areas of
machine learning that involve multidimensional data such as
analysis of texts (Peng et al., 2011), bioinformatics (Sato and
Sakakibara, 2005; Liu et al., 2006), computer vision (Brecht et
al., 2004; Sminchisescu et al., 2006), and natural language
processing (Lafferty et al., 2001; Choi et al., 2005), but their
application to identify neurons with pattern completion capa-
bility that could allow the study of the properties and mecha-
nisms of neuronal ensemble formation is lacking.
Population vectors representing the coordinated activity of
neuronal groups were inferred from calcium imaging recordings
and used as training data. We defined activity events from each
neuron as nodes in an undirected graph, where each node can
have two values, 0 corresponding to nonactivity and 1 corre-
sponding to activity. In this way, nodes interact with each other
through connecting edges, which have four possible configura-
tions, 00, 01, 10, and 11, depending on the values of the two
nodes adjacent to the edge. The two values associated with nodes
Carrillo-Reid et al. · Identification of Pattern Completion Neurons
before imprinting after imprinting
A
B
C 1 1
0.8 0.8
0.6 0.6
AUC
AUC
0.4 0.4
0.2 0.2
0 0
-1.2 -0.8 -0.4 -1.2 -0.8 -0.4
node strength node strength
Figure 4. Confirming pattern completion with two-photon optogenetics. A, Graphical
models obtained using CRFs from simultaneous two-photon imaging and two-photon opto-
genetic stimulation of a neuron with pattern completion capability before (left) and after
(right) two-photon optogenetic ensemble imprinting. Ensemble connections are highlighted
with blue lines. Scale bar, 50 mm. B, Circular graphs of CRF models before (pre-) and after
(post-) ensemble imprinting. Connections between ensemble neurons are shown in blue.
Red represents photo-stimulated neuron (arrow). C, Node strength and AUC values showed
network changes of neurons with pattern completion capability. The photo-stimulated neu-
ron is represented in red before (left) and after (right) ensemble imprinting. Confidence lev-
els calculated from random data are depicted by gray bars.
and the four values associated with edges are characterized by a set
of parameters called node potentials ð f 0 ; f 1 Þ and edge potentials
ð f 00 ; f 01 ; f 10 ; f 11 Þ respectively (Fig. 2B). These parameters, also
known as potential functions, reflect the scores of individual values
on each node and edge. Using part of the observation data (e.g.,
training data), we estimated the model’s structure (e.g., graph con-
nectivity) and parameters (e.g., the potentials) via maximum likeli-
hood (see above, Materials and Methods). We then performed
cross-validation on held-out data to identify the optimal level of
sparsity and regularization (see above, Materials and Methods).
Once the model was learned from the data, the normalized product
of the corresponding nodes and edge potentials describe a probabil-
ity distribution that gives a scalar score when a given neuronal pop-
ulation becomes active. This is a normalized probability over all
possible network states (e.g., all binary configurations of the nodes)
and is estimated by maximizing likelihood on past observed popula-
tion vectors (see above, Materials and Methods).
To integrate information about the external stimulus along
with the observed neuronal data, we added an additional node
for each type of stimulus that was presented to the animal. This
Carrillo-Reid et al. · Identification of Pattern Completion Neurons J. Neurosci., October 13, 2021 • 41(41):8577–8588 • 8583

 
A p xm jym ; ys1 ¼ 0; ys2 ¼ 1 . Thus, assum-
ing that the a priori probability of visual
stimuli is uniform, the higher probability
ensemble 1

ensemble 2

ensemble 3

ensemble 4
5s value through comparing the log-likeli-
hood ratio logðps1 Þ  logðps2 Þ can be used
to classify each stimuli (Fig. 2D). To quan-
tify the classification performance of
CRFs, we calculated the accuracy percent-
B age in experimental data and compared it
1 with shuffled models in which node poten-
tials and edge potentials from the real
ensemble

2 model were randomly shuffled (Fig. 2E).

The accuracy of real models was signifi-
3
cantly higher than shuffled models (n = 8
mice; real model accuracy = 79.82 6
4
8.34%; shuffled model accuracy = 61.21 6
22.55%; mean 6 SD; p , 0.0001; Mann–
C D Whitney test) demonstrating that CRFs
1 1
1 * can model different orientations of drifting
**
*** gratings from calcium imaging population
0.8
recordings.
AUC

TF=1 0.6
TPR

TPR

0.5 TF=2 0.5

0.4 Identification of neurons with pattern
TF=4
TF=8
completion properties
0.2
TF=15 After the successful generation of CRF
0 0 0 models that describe different experimen-
0 0.5 1 0 0.5 1 1 2 4 8 15
TF (Hz)
tal conditions (vertical or horizontal drift-
FPR FPR ing-gratings; Fig. 2D) we investigated
E whether the parameters obtained from
1 1
such models could be used to identify
neurons with putative pattern comple-
tion properties. To identify pattern com-
TPR

TPR

0.5 0.5 pletion neurons we set the activity of each

neuron to 1 or 0 and compared the output
logarithm probability difference (pro-  m
bability ratio) logpm  logpm i;0 ¼ logp y j
0 0 i;1
0 0.5 1 0 0.5 1 x=i ; xi ¼ 1Þ  logp ym jxm
m m
;
=i ix m
¼ 0 using
FPR FPR
the inferred CRF models (Fig. 3A). In other
Figure 5. Applicability of CRFs to public calcium imaging data. A, Calcium transients of representative pattern completion neurons words, we tested how good each neuron is to
identified from CRF models for different orientations of drifting gratings. Gray stripes indicate visual stimuli for the corresponding orien- predict each model. We calculated single neu-
tation of drifting gratings for each neuronal ensemble. B, Temporal course of neuronal ensemble activation. Each neuronal ensemble ron performance by binarizing the logarithm
was defined by all putative pattern completion neurons with high AUC and high node strength for each orientation of drifting gratings. probability difference (Fig. 3B) and calculated
Each row shows the activation of a specific neuronal ensemble (black lines). Scale bar, 200 frames. C, ROC curves of neuronal ensembles the area under the ROC curve (AUC).
predicting different temporal frequencies (TF: 1, 2, 4, 8, and 15 Hz). Dashed line represents random classification performance. D, AUC Because pattern completion neurons are
for different TFs (classification AUC: TF 1 = 0.7906 6 0.0301, TF 2 = 0.7158 6 0.0407, TF 4 = 0.7009 6 0.0323, TF 8 = 0.6212 6
likely to have strong functional connectivity,
0.0335, TF 15 = 0.5642 6 0.0288; p1,2 = 0.2277 ns; p1,4 = 0.0423*; p1,8 = 0.001**; p1,15 = 4.2583e-05***). Note that prediction per-
formance of CRF models with TF = 1 decreases with data with different TFs. E, Preferred orientation selectivity of neuronal ensembles we used the node strength obtained from
for TF = 1 Hz. The radius of each circle depicts AUC values from 0 (center) to 1 (border). Dotted inner circles represent random perform- CRF models that represents the summation
ance (AUC = 0.5; n = 6 mice, 24 ensembles; Wilcoxon rank sum test). of edge potentials (f 11 terms) from all con-
necting edges for each node in the graph. In
this way, strongly connected neurons have
node was set to 1 when the corresponding stimulus was on and 0 high node strength, whereas weakly con-
when the stimulus was off (Fig. 2B). In this way, CRFs modeled nected neurons have low node strength (Fig. 3C). Finally, we defined
the conditional probability of network states given the observa- putative pattern completion neurons as the ones that can be used to
tions for different visual stimuli. Therefore, by treating visual predict each experimental condition with higher performance (high
stimuli as added nodes and comparing the output likelihood of AUC) and have high node strength (Fig. 3D). It is important to high-
observing each stimulus, CRFs were able to model different light that some neurons could take part in several ensembles simulta-
visual stimuli from observed data. The nodes directly connected neously (Carrillo-Reid et al., 2015b). Such neurons that respond to
to the added nodes generate a specific model for each experimen- different experimental conditions have high functional connectivity
tal condition (Fig. 2C). Given two different visual stimuli (hori- but are not able to recall a specific ensemble (Carrillo-Reid et al.,
zontal or vertical drifting gratings; ys1 and ys2, respectively), the 2019), therefore the use of AUC is fundamental to distinguish neu-
logarithm of probability corresponding
 to observing each stimu- rons with high functional connectivity that can recall a specific en-
lus is defined by ps1 ¼ p xm jym ; ys1 ¼ 1; ys2 ¼ 0 and ps2 ¼ semble. Our analysis shows that neurons with putative pattern
8584 • J. Neurosci., October 13, 2021 • 41(41):8577–8588 Carrillo-Reid et al. · Identification of Pattern Completion Neurons

completion capability have significantly A

higher AUC compared with nonpattern 0.6 0.6
completion (non-p.c.) neurons that have ******* ***

similarity

similarity
high node strength [Fig. 3E; n = 14 mice; 0.4 0.4
AUC p.c. neurons = 0.8063 6 0.1277;
AUC non-p.c. neurons = 0.2520 6 0.2 0.2

0.1649; mean 6 SD; p , 0.0001; Mann– 0 0

Whitney test]. Noticeably, not all the 20 60 100 140 180 20 60 100 140 180

neurons with high node strength are neurons from CRFs (%) neurons from CRFs (%)
able to predict the given stimuli.
B
1
Optogenetic confirmation of pattern ******* 1

completion prediction using CRF 0.8 0.8

AUC

AUC
models
We then tested the CRF prediction of pu- 0.6 0.6
tative pattern completion neurons using
previously published data, where artifi- 0.4
20 60 100 140 180 0.4
20 60 100 140 180
cially imprinted cortical ensembles were
created by the repetitive activation of a neurons from CRFs (%) neurons from CRFs (%)
group of neurons with two-photon opto-
genetics and recalled by activating specific C 1 optimal ensemble 1
members of the ensemble, a property
known as pattern completion (Carrillo-
0.8 0.8
Reid et al., 2016). To do so, we used 180 180

neurons from CRFs (%)

the structural and classification parame- 160

random neurons (%)

160
0.6 140 0.6
ters of CRFs learned from simultaneous
TPR

TPR
140
120
two-photon imaging and two-photon 120
0.4 100 0.4 100
optogenetic experiments with single-cell 80 80
resolution before and after the imprinting 60 60
protocol and demonstrated that pattern 0.2 40 0.2
40
completion neurons increased their con- 20 20
nectivity after the imprinting protocol 0
0 0.5 1
0
0 0.5 1
(Fig. 4A). To clearly visualize the change
FPR FPR
in graph connectivity induced by the
imprinting protocol, we constructed iso- Figure 6. CRF efficacy to model pattern completion neurons from cortical ensembles. A, Cosine similarity from putative pat-
morphic graphs from the CRF models tern completion neurons extracted from CRFs that have been down sampled or up sampled (left) or randomly sampled (right).
and arranged them in a circular visualiza- Black triangle indicates the original ensemble size. Note that removing or adding neurons decreases the ability to predict visual
tion (Fig. 4B). After the artificial ensemble stimuli. B, AUC values of down-sampled or up-sampled putative pattern completion neurons (left) or randomly sampled neu-
was imprinted, neurons with pattern rons (right). C, ROC curves of down-sampled or up-sampled neurons (left) or randomly sampled neurons (right; n = 8 mice, 28
completion capability that were able to ensembles; Wilcoxon rank sum test).
recall an entire ensemble showed signifi-
cantly increased AUC (Fig. 4C; n = 6 concomitant responses to the corresponding orientation (Fig.
mice; AUC before = 0.4570 6 0.1256; AUC after = 0.8568 6 5A). Putative pattern completion neurons were also able to
0.0952; mean 6 SD; p = 0.0313; Wilcoxon matched-pairs signed- define specific neuronal ensembles for each orientation (Fig. 5B).
rank test). These analyses demonstrate that structural and classifi- The ROC curves comparing the performance of the model with
cation parameters inferred with CRFs can be used not only to TF = 1 against data obtained with the same orientation but with
identify pattern completion neurons but also to study the mecha- different TFs showed that our approach could also be used to
nisms involved in the reconfiguration and recalling of neuronal detect changes in specific features of visual stimuli in layer 2/3 of
ensembles from optogenetically perturbed microcircuits. primary visual cortex as different TFs showed a degraded classifi-
cation performance (Fig. 5C,D; classification AUC: AUCTF=1 =
Applicability of CRF models to publicly available datasets 0.79 6 0.15, AUCTF=2 = 0.71 6 0.21, AUCTF=4 = 0.69 6 0.17,
One of the challenges for the identification of putative pattern AUCTF=8 = 0.62 6 0.17, AUCTF=15 = 0.56 6 0.14; p1,4 , 0.05,
completion neurons consists in the applicability of the methods p1,8 , 0.01, p1,15 , 0.001; n = 5 animals, 20 ensembles; mean 6
to data obtained in different laboratories under diverse experi- SD; Wilcoxon rank sum test). Therefore, our approach was able
mental conditions. To explore the general applicability of our to find neurons with putative pattern completion properties,
method, we analyzed publicly open datasets (Allen Brain from a publicly open dataset, that are tuned to a particular orien-
Observatory), which contain data from layer 2/3 of primary vis- tation of drifting gratings for a given TF (Fig. 5E), suggesting
ual cortex consisting in several visual stimuli types with different that CRFs could be used to recall percepts with varied features.
experimental settings. We generated CRF models for each of
four orientations of drifting gratings at a temporal frequency Quantifying CRF prediction performance for ensembles
(TF) of 1 and compared the performance of the model for the We next investigated whether the neurons highlighted by CRFs
same orientation but with different TFs. For each model, neu- were optimal for predicting visual stimuli. To do so, we ran-
rons with high AUCs and high node strength showed domly shuffled population vectors containing all putative pattern
Carrillo-Reid et al. · Identification of Pattern Completion Neurons J. Neurosci., October 13, 2021 • 41(41):8577–8588 • 8585

A B lower than those of CRF models (best sim-

sorted nets (simulation) 0.6 ilarity 0.1000 6 0.0101, AUC 0.5045 6
0.0071; mean 6 SEM), indicating that

probability
neurons identified with CRF parameters
0.4
achieved a classification performance sig-
neurons

in vivo
nificantly better than random sets of neu-
0.2 rons. These results suggest that targeting
all the neurons that were active at different
trials of a given experimental condition
0
4 5 6 7 8 9 10 could reduce the accuracy of recalled
time active neurons (%) ensembles.
C D
1 Modeling CRFs with different
cumulative probability

1
parameters using simulated neural
score (simulation)
networks
To test the performance of CRFs in model-
shuffled

ing the pattern completion properties of

simulated neural networks, we embedded
10 Hopfield networks within a larger net-
work of weaker connectivity (Fig. 7A; see
0 0
0.5 0.6 0.7 0.8 0.9 1 0.2 0.4 0.6 0.8 1 above, Materials and Methods). To achieve
similarity score (in vivo) biologically plausible population vectors,
we constructed our network so that the
E F mean of neuronal activity following stim-
1 1 uli presentation matched the mean of neu-
p.c. neurons
score (simulation)

ronal activity of high-activity frames in our

0.8
correlation

in vivo datasets (Fig. 7B). To replicate a plau-

non-p.c. neurons

0.6
**** sible level of underdetermination, we mod-
eled this simulated network at eight values
0.4
of the structure-learning regularization term,
0.2 which is a regularization parameter that con-
0 trols the sparsity of the graphical structure
10 20 30 40
neurons
0 l s (7.9767e-05-0.1793), and two values
of the parameter-learning regularization
Figure 7. Pattern completion neurons in simulated networks. A, Raster-plot of simulated data sorted by Hopfield networks term, which controls the learned parameters
(100 ensembles from 10 Hopfield nets). B, Probability distribution of simulated network. The network’s response to trained and helps in preventing overfitting, l p (10,
stimuli was comparable to the level of activity observed in high-activity frames in our in vivo datasets (dashed line). 10,000). First, we assessed how accurately
Population activity was defined as the percentage of neurons active during any given frame. C, The distribution of cosine sim- our model could describe the functional
ilarities comparing the weights of the simulated network and CRF model for each neuron. Dash line denotes shuffled data. connectivity of our simulated network. To
D, Top-ranked neurons according to their score in both simulated network and CRFs. Correlation coefficient and linear regres-
accomplish this, we used cosine similarity to
sion of scores calculated from the simulated network and CRF model ( r = 0.8592; r2 = 0.738). E, Comparison between pu-
tative pattern completion neurons identified in simulated networks and CRFs for different set sizes. F, Pattern completion
measure the synaptic weight matrix of the
neurons from each ensemble have higher scores than nonpattern completion neurons (n = 100 ensembles from 10 Hopfield simulated network with the edge potentials
networks; p.c. neurons score = 0.7891 6 0.0452; non-p.c. neurons score = 0.4794 6 0.0399; p , 0.0001 (****); mean 6 of our model. We found a striking similarity
SD; Mann–Whitney test). between the synaptic weights of each neuron
and respective edge potentials in the CRFs
(cosine similarity = 0.9615 6 0.0017; mean
completion neurons from cortical ensembles by adding or 6 SEM), which was not observed in a shuf-
removing elements from the group and examined the stimulus fled model (cosine similarity = 0.0052; Fig. 7C). Thereafter, we
prediction performance. The similarity function (Fig. 6A, left), investigated whether using the real connectivity of the simulated
prediction performance (Fig. 6B, left) and ROC curves (Fig. 6C, network or the learned connectivity of the model would identify the
left) of population vectors formed with detected cortical ensem- same pattern completion neurons. To compare performance using
bles had a maximum value when the neurons depicted by CRFs one metric, we averaged the scores for prediction performance
were unchanged (original neurons similarity 0.2147 6 0.0206; (AUC) and normalized node strength. We found the correlation
prediction AUC 0.8109 6 0.0242; mean 6 SEM). These results between real and modeled ensembles to be very strong (r2 = 0.7380,
showed that population vectors formed with neurons identified r = 0.8592; Fig. 7D). To determine whether both methods would
by CRFs represent an efficient population to predict external vis- identify the same neurons for targeted stimulation, we compared
ual stimuli. This raises the question of whether such population sets of neurons selected by each method using the Jaccard index.
vectors are a specific nonrandom subgroup. To answer this, we We found that both methods targeted similar groups of pattern
randomly sampled a subset from all the population of neurons completion neurons, and this relationship was maintained across
ranging from 10 to 190% of the total size of neuronal ensembles. variations in set size (Fig. 7E). Our analysis of these targeted groups
Indeed, we observed that the similarity function (Fig. 6A, right), confirmed that a handful of neurons in each ensemble possessed
prediction performance (Fig. 6B, right) and ROC curves (Fig. pattern completion capability demonstrated by significantly higher
6C, right) from random groups of neurons was significantly scores between pattern completion neurons and nonpattern
8586 • J. Neurosci., October 13, 2021 • 41(41):8577–8588 Carrillo-Reid et al. · Identification of Pattern Completion Neurons

completion neurons (Fig. 7F; n = 100 A B

ensembles; p.c. neurons score = 0.7891 6 16k
600
small
0.0452; non-p.c. neurons score = 0.4794 6 small

CPU time (seconds)

CPU time (seconds)

medium medium
0.0399; p , 0.0001; mean 6 SD; Mann– 12k large
450
large

Whitney test), supporting the hypothesis

that neurons with pattern completion capa- 8k 300
bility have increased functional connectivity
(Carrillo-Reid et al., 2016, 2019). 4k 150

0 0
Discussion 0 2500 5000 7500 10000 0 2500 5000 7500 10000

Graph theory analysis of functional population vectors (frames) population vectors (frames)
connectivity in cortical microcircuits Figure 8. Modeling CRFs with different parameters. A, Run time of structural learning across small (n = 64), medium
In this study, we use a computational (n = 500), and large (n = 1000) datasets of increasing size. Each run generated 100 structures using l values between 1e-
s
method, CRFs, to identify pattern comple- 5 and 0.5. With datasets of 1000 neurons and 10 000 population vectors (prescreened for the inclusion of only high-activity
tion neurons in calcium imaging recordings frames), run times did not surpass 5 h. B, Mean run time of parameter learning for individual models using small (n = 64),
of mouse visual cortex in vivo. As opposed to medium (n = 500), and large (n = 1000) datasets of increasing size. With datasets of 1000 neurons and 10,000 population
traditional descriptive approaches for neuro- vectors (prescreened for the inclusion of only high-activity frames), run times did not surpass 10 min per model.
nal ensemble identification and network Approximately one-half of this run time comprised block-coordinate Frank–Wolfe iterations. The remainder of this run time
analyses, based on correlations between pairs can be attributed to the initiation of each model and frequent reporting of progress.
of neurons, machine learning methods repre-
sent an empirically grounded new approach The computational difficulty in constructing CRFs lies in
to create models that aim to capture the functional structure of neu- recovering the global normalizer (the partition function) and the
ronal circuits while providing structural information about the net- corresponding gradients. With an arbitrary graph structure, this
work properties of individual neurons within a population. problem is often intractable. But recent advances that combine
In the past decades, graph theory has been applied to charac- Bethe free energy approximation and Frank–Wolfe methods for
terize the structure and function of neuronal networks (Achard inference and learning model parameters allow fast and a rela-
and Bullmore, 2007; Bettencourt et al., 2007; Fair et al., 2008; tively accurate construction of cyclic CRFs (Tang et al., 2016).
Hagmann et al., 2008; Iturria-Medina et al., 2008; Supekar et al., Thus, CRFs could be applied to datasets with thousands of inter-
2008; Yu et al., 2008; Downes et al., 2012; Zuo et al., 2012; connected neurons. However, for datasets with more neurons
Cunningham and Yu, 2014; Chiang et al., 2016). Most of these (and therefore more random variables and larger networks),
studies used recordings across multiple brain regions (Achard CRFs (like most machine learning approaches) would require an
and Bullmore, 2007; Fair et al., 2008; Zuo et al., 2012; Hinne et increasingly large number of samples (population vectors) in the
al., 2013; Chiang et al., 2016), and only a few focused on the gen- training dataset.
eral network properties of cortical circuits with recordings from
single neurons (Bonifazi et al., 2009; Stetter et al., 2012; Advantages and limitations of CRFs compared with other
Gururangan et al., 2014; Yatsenko et al., 2015). The majority of classification algorithms
methods applied to infer network properties in brain slices (Mao Compared with fully generative models such as Markov random
et al., 2001; Cossart et al., 2003; Ikegaya et al., 2004; Stetter et al., fields and Bayesian networks that make assumptions on the
2012; Gururangan et al., 2014) or in vivo (Yatsenko et al., 2015) dependencies among all the observed variables from the model,
operate on the correlation matrix and aim to recover the CRFs only model the hidden system states dependent on
functional dependencies between observed pairs of neurons. observed features. Because no independence assumptions are
However, these methods are model-free and therefore incapable made among observed variables, CRFs avoid potential errors
of describing the overall network dynamics based on the proba- introduced by unobserved common inputs. Additionally, given
bility distribution of neuronal ensembles. Our method provides the finite number of network states described by population ac-
an alternative by directly modeling the statistical dependencies tivity, the conditional distribution is enough for making predic-
among all the neurons. The generation of a graphical model of tions and for identifying neurons in each state. Compared with
the circuit provides a direct link to the functional connectivity other discriminative finite-state models such as the maximum
structure of the circuit, enabling targeted manipulations of com- entropy Markov model (MEMM), CRFs use global normalizers
ponents, as opposed to dimensionality reduction or encoding to overcome the local bias in MEMM, induced by local normal-
methods. izers, and achieve higher accuracy in diverse applications
(Lafferty et al., 2001). Also, compared with encoders that adjust a
CRFs graphical models identify pattern completion neurons series of constants associated with each neuron, CRFs can be
in neuronal ensembles used to explore changes in functional connectivity patterns (as
Cortical ensembles represent coactive neuronal populations that we demonstrate here) and their relation to learning or behavioral
could represent the computational unit of brain functions (Mao states. Therefore, CRFs appear promising for modeling neuronal
et al., 2001; Bonifazi et al., 2009; Miller et al., 2014; Carrillo-Reid ensemble functional connectivity and for identifying members
et al., 2015a, 2016, 2019). The structural and functional organiza- that could be easily manipulated by two-photon optogenetics.
tion of brain microcircuits could be characterized by these mod- The overall activity of multiple cells in a given time window
ular properties. For this reason, it is necessary to identify pattern can be understood as a multidimensional array of population
completion neurons in these cortical modules that can efficiently vectors where vectors pointing to a similar location in space can
recall different brain states (Gururangan et al., 2014; Carrillo- be considered as a group. We previously showed that population
Reid et al., 2015a). vectors defining a group (i.e., a neuronal ensemble) can be
Carrillo-Reid et al. · Identification of Pattern Completion Neurons
extracted from multidimensional arrays by performing singular
value decomposition (SVD; Carrillo-Reid et al., 2015a). Although
SVD can identify cortical ensembles reliably, it lacks a structured
graphical model that allows the systematic study of changes in net-
work properties and the identification of neurons with pattern
completion capability. One limitation for the current CRF learning
algorithm is the computation of a sparse graph structure, which is
less prone to overfitting of the data. Certainly, because of some
modeling assumptions, computational considerations and the
finiteness of training data, the learned graphical structure and pa-
rameters may not be the globally best probabilistic model of the
data. However, it is recovered efficiently rather than requiring an
exhaustive and unrealistic exploration of all possible structures
and parameter combinations. Additionally, approximations dur-
ing the parameter learning step can sometimes compromise the
global optimality guarantees.
Applicability of CRFs to broader datasets
To assess the scalability of CRFs, we also performed benchmark-
ing tests using sample artificial networks of 64, 500, and 1000
neurons with up to 10,000 frames of activity. We found that 100
l s runs completed in ,270 min of CPU time when using the
largest dataset (Fig. 8A). Parameter estimation was trivial in
comparison, requiring just 600 s of CPU time per model at the
largest dataset (Fig. 8B). Both structural learning and parameter
estimation can be easily parallelized by neighborhood (structural
learning) and by model (parameter estimation), leading to highly
efficient wall clock run times. Furthermore, given the sequential
execution of structural learning and parameter estimation, it is
also possible to substitute more efficient methods of structural
learning, such as Pearson’s correlation, to further prune wall
clock run times. Given the efficient wall clock run times, it is fea-
sible for future research to consider expanding our approach to
incorporate multiframe ensembles and other temporal interac-
tions or use continuous (instead of binarized) neural data by
replacing the linear programming solver used during parameter
estimation. Finally, given the efficiency of parameter estimation,
the development of equally efficient structural learning methods
will allow for the analysis of streaming network data.
Identification of pattern completion neurons for optogenetic
targeting
One key advantage of CRFs is the ability to model the circuit ex-
plicitly in a manner than can be used for targeted manipulation.
Indeed, the results from the present work (Fig. 4) and those from
our previous publication (Carrillo-Reid et al., 2019), demonstrate
that neurons with high node strength and AUC values are partic-
ularly prone to be pattern completion neurons. As one practical
example of an application of this method, nontargeted electrical
stimulation of visual cortex has been used for decades as an
attempt to provide useful visual sensations to patients who have
lost the functionality of their eyes (Brindley and Lewin, 1986).
To improve prostheses, one could in principle train patients
using devices with a large number of electrodes (Shepherd et al.,
2013). Our results suggest that after a given network has been
imprinted (Carrillo-Reid et al., 2016), the identification of neu-
rons with pattern completion capability could be used to reduce
the number of active points that require stimulation and pin-
point them with accuracy. The further development of network
models based on population activity that can predict a given set
of features embedded in visual stimuli will be crucial for the effi-
cient manipulation of cortical ensembles.
J. Neurosci., October 13, 2021 • 41(41):8577–8588 • 8587
Finally, it has been shown that the connectivity of diverse sys-
tems described by graphs with complex topologies follow a scale-
free power law distribution (Barabasi and Albert, 1999).
Similarly, cortical ensembles described by CRFs could be charac-
terized by a subset of neurons with strong synaptic connections.
The existence of neurons with pattern completion capability has
been demonstrated in previous studies where perturbing the ac-
tivity of single neurons was able to change the overall network
dynamics (Hagmann et al., 2008; Bonifazi et al., 2009; Carrillo-
Reid et al., 2016).
In closing, our results suggest that the structural and predic-
tive parameters defined by CRF models could be used in the
design of closed-loop experiments with single-cell resolution to
investigate the role of a specific subpopulation of neurons in a
given cortical microcircuit during different behavioral events
(Carrillo-Reid et al., 2019).
References
Achard S, Bullmore E (2007) Efficiency and cost of economical brain func-
tional networks. PLoS Comput Biol 3:e17.
Barabasi AL, Albert R (1999) Emergence of scaling in random networks.
Science 286:509–512.
Bettencourt LM, Stephens GJ, Ham MI, Gross GW (2007) Functional struc-
ture of cortical neuronal networks grown in vitro. Phys Rev E Stat Nonlin
Soft Matter Phys 75:021915.
Bonifazi P, Goldin M, Picardo MA, Jorquera I, Cattani A, Bianconi G, Represa
A, Ben-Ari Y, Cossart R (2009) GABAergic hub neurons orchestrate syn-
chrony in developing hippocampal networks. Science 326:1419–1424.
Brecht M, Schneider M, Sakmann B, Margrie TW (2004) Whisker move-
ments evoked by stimulation of single pyramidal cells in rat motor cortex.
Nature 427:704–710.
Brindley GS, Lewin WS (1986) The sensations produced by electrical stimula-
tion of the visual cortex. J Physiol 196:479–493.
Brown SL, Joseph J, Stopfer M (2005) Encoding a temporally structured stim-
ulus with a temporally structured neural representation. Nat Neurosci
8:1568–1576.
Bullmore E, Sporns O (2009) Complex brain networks: graph theoretical analysis
of structural and functional systems. Nat Rev Neurosci 10:186–198.
Carrillo-Reid L, Tecuapetla F, Tapia D, Hernández-Cruz A, Galarraga E,
Drucker-Colin R, Bargas J (2008) Encoding network states by striatal cell
assemblies. J Neurophysiol 99:1435–1450.
Carrillo-Reid L, Lopez-Huerta VG, Garcia-Munoz M, Theiss S, Arbuthnott
GW (2015a) Cell assembly signatures defined by short-term synaptic
plasticity in cortical networks. Int J Neural Syst 25:1550026.
Carrillo-Reid L, Miller JE, Hamm JP, Jackson J, Yuste R (2015b) Endogenous
sequential cortical activity evoked by visual stimuli. J Neurosci 35:8813–
8828.
Carrillo-Reid L, Yang W, Bando Y, Peterka DS, Yuste R (2016) Imprinting
and recalling cortical ensembles. Science 353:691–694.
Carrillo-Reid L, Yang W, Kang Miller JE, Peterka DS, Yuste R (2017)
Imaging and optically manipulating neuronal ensembles. Annu Rev
Biophys 46:271–293.
Carrillo-Reid L, Han S, Yang W, Akrouh A, Yuste R (2019) Controlling visu-
ally guided behavior by holographic recalling of cortical ensembles. Cell
178:447–457.e5.
Carrillo-Reid L, Yuste R (2020a) Playing the piano with the cortex: role of
neuronal ensembles and pattern completion in perception and behavior.
Curr Opin Neurobiol 64:89–95.
Carrillo-Reid L, Yuste R (2020b) What is a neuronal ensemble? In Oxford
Research Encyclopedia of Neuroscience (Sherman SM ed), pp 1–23.
Oxford: Oxford UP.
Chiang S, Cassese A, Guindani M, Vannucci M, Yeh HJ, Haneef Z, Stern JM
(2016) Time-dependence of graph theory metrics in functional connec-
tivity analysis. Neuroimage 125:601–615.
Choi Y, Cardie C, Riloff E, Patwardhan S (2005) Identifying sources of opin-
ions with conditional random fields and extraction patterns. Paper pre-
sented at Conference on Human Language Technology and Empirical
Methods in Natural Language Processing, Vancouver, British Columbia,
Canada, October.
8588 • J. Neurosci., October 13, 2021 • 41(41):8577–8588
Cossart R, Aronov D, Yuste R (2003) Attractor dynamics of network UP
states in the neocortex. Nature 423:283–288.
Cunningham JP, Yu BM (2014) Dimensionality reduction for large-scale
neural recordings. Nat Neurosci 17:1500–1509.
de Vries SEJ, Lecoq JA, Buice MA, Groblewski PA, Ocker GK, Oliver M,
Feng D, Cain N, Ledochowitsch P, Millman D, Roll K, Garrett M,
Keenan T, Kuan L, Mihalas S, Olsen S, Thompson C, Wakeman W,
Waters J, Williams D, et al. (2020) A large-scale standardized physiologi-
cal survey reveals functional organization of the mouse visual cortex. Nat
Neurosci 23:138–151.
Downes JH, Hammond MW, Xydas D, Spencer MC, Becerra VM, Warwick
K, Whalley BJ, Nasuto SJ (2012) Emergence of a small-world functional
network in cultured neurons. PLoS Comput Biol 8:e1002522.
Drew PJ, Shih AY, Driscoll JD, Knutsen PM, Blinder P, Davalos D,
Akassoglou K, Tsai PS, Kleinfeld D (2010) Chronic optical access through
a polished and reinforced thinned skull. Nat Methods 7:981–984.
Eguíluz VM, Chialvo DR, Cecchi GA, Baliki M, Apkarian AV (2005) Scale-
free brain functional networks. Phys Rev Lett 94:018102.
Fair DA, Cohen AL, Dosenbach NU, Church JA, Miezin FM, Barch DM,
Raichle ME, Petersen SE, Schlaggar BL (2008) The maturing architecture
of the brain’s default network. Proc Natl Acad Sci U S A 105:4028–4032.
Gerstner W, Kistler WM, Naud R, Paninski L (2014) Memory and attractor
networks. In Network Dynamics: From singles neurons to networks and
models of cognition and beyond, pp 443–457. Cambridge: Cambridge
UP.
Gururangan SS, Sadovsky AJ, MacLean JN (2014) Analysis of graph invari-
ants in functional neocortical circuitry reveals generalized features com-
mon to three areas of sensory cortex. PLoS Comput Biol 10:e1003710.
Hagmann P, Cammoun L, Gigandet X, Meuli R, Honey CJ, Wedeen VJ,
Sporns O (2008) Mapping the structural core of human cerebral cortex.
PLoS Biol 6:e159.
Hebb DO (1949) The organization of behavior; a neuropsychological theory.
New York: Wiley.
Hinne M, Heskes T, Beckmann CF, van Gerven MAJ (2013) Bayesian infer-
ence of structural brain networks. NeuroImage 66:543–552.
Hopfield JJ (1982) Neural networks and physical systems with emergent col-
lective computational abilities. Proc Natl Acad Sci U S A 79:2554–2558.
Ikegaya Y, Aaron G, Cossart R, Aronov D, Lampl I, Ferster D, Yuste R (2004)
Synfire chains and cortical songs: temporal modules of cortical activity.
Science 304:559–564.
Iturria-Medina Y, Sotero RC, Canales-Rodríguez EJ, Alemán-Gómez Y,
Melie-García L (2008) Studying the human brain anatomical network via
diffusion-weighted MRI and graph theory. Neuroimage 40:1064–1076.
Khazaee A, Ebrahimzadeh A, Babajani-Feremi A (2015) Identifying patients
with Alzheimer’s disease using resting-state fMRI and graph theory. Clin
Neurophysiol 126:2132–2141.
Ko H, Hofer SB, Pichler B, Buchanan KA, Sjöström PJ, Mrsic-Flogel TD
(2011) Functional specificity of local synaptic connections in neocortical
networks. Nature 473:87–91.
Koller D, Friedman N (2009) Probabilistic graphical models: principles and
techniques. Cambridge, MA: MIT press.
Lafferty J, McCallum A, Pereira F (2001) 18th International Conference on
Machine Learning, pp 282–289. San Francisco (ICML), CA: Morgan
Kaufmann.
Liu Y, Carbonell J, Weigele P, Gopalakrishnan V (2006) Protein fold recogni-
tion using segmentation conditional random fields (SCRFs). J Comput
Biol 13:394–406.
Lorente de Nó R (1938) Analysis of the activity of the chains of internuncial
neurons. J Neurophysiol 1:207–244.
Mao BQ, H-S F, Aronov D, Froemke RC, Yuste R (2001) Dynamics of spon-
taneous activity in neocortical slices. Neuron 32:883–898.
Marr D (1971) Simple memory: a theory for archicortex. Philos Trans R Soc
Lond B Biol Sci 262:23–81.
Marshel JH, Kim YS, Machado TA, Quirin S, Benson B, Kadmon J, Raja C,
Chibukhchyan A, Ramakrishnan C, Inoue M, Shane JC, McKnight DJ,
Yoshizawa S, Kato HE, Ganguli S, Deisseroth K (2019) Cortical layer-spe-
cific critical dynamics triggering perception. Science 365:eaaw5202.
Micheloyannis S, Vourkas M, Tsirka V, Karakonstantaki E, Kanatsouli K,
Stam CJ (2009) The influence of ageing on complex brain networks: a
graph theoretical analysis. Hum Brain Mapp 30:200–208.
Carrillo-Reid et al. · Identification of Pattern Completion Neurons
Miller JE, Ayzenshtat I, Carrillo-Reid L, Yuste R (2014) Visual stimuli recruit
intrinsically generated cortical ensembles. Proc Natl Acad Sci U S A 111:
E4053–4061.
Mukamel EA, Nimmerjahn A, Schnitzer MJ (2009) Automated analysis of cellu-
lar signals from large-scale calcium imaging data. Neuron 63:747–760.
Packer AM, Peterka DS, Hirtz JJ, Prakash R, Deisseroth K, Yuste R (2012)
Two-photon optogenetics of dendritic spines and neural circuits. Nat
Methods 9:1202–1205.
Packer AM, Russell LE, Dalgleish HW, Hausser M (2015) Simultaneous all-
optical manipulation and recording of neural circuit activity with cellular
resolution in vivo. Nat Methods 12:140–146.
Peng H-K, Zhu J, Piao D, Yan R, Zhang Y (2011) Retweet modeling using
conditional random fields. Paper presented at the IEEE 11th
International Conference on Data Mining Workshops, Vancouver,
British Columbia, Canada, December.
Ravikumar P, Wainwright MJ, Lafferty JD (2010) High-dimensional Ising model
selection using ‘1-regularized logistic regression. Ann Statist 38:1287–1319.
Rickgauer JP, Deisseroth K, Tank DW (2014) Simultaneous cellular-resolu-
tion optical perturbation and imaging of place cell firing fields. Nat
Neurosci 17:1816–1824.
Robinson NTM, Descamps LAL, Russell LE, Buchholz MO, Bicknell BA,
Antonov GK, Lau JYN, Nutbrown R, Schmidt-Hieber C, Häusser M
(2020) Targeted activation of hippocampal place cells drives memory-
guided spatial behavior. Cell 183:1586–1599.e10.
Rolls ET, Treves A (1994) Neural networks in the brain involved in memory
and recall. Prog Brain Res 102:335–341.
Sasaki T, Matsuki N, Ikegaya Y (2007) Metastability of active CA3 networks.
J Neurosci 27:517–528.
Sato K, Sakakibara Y (2005) RNA secondary structural alignment with condi-
tional random fields. Bioinformatics 21 Suppl 2:ii237–242.
Schaub MT, Schultz SR (2012) The Ising decoder: reading out the activity of
large neural ensembles. J Comput Neurosci 32:101–118.
Schreiber S, Fellous JM, Whitmer D, Tiesinga P, Sejnowski TJ (2003) A new
correlation-based measure of spike timing reliability. Neurocomputing
52-54:925–931.
Shepherd RK, Shivdasani MF, Nayagam DF, Williams CF, Blamey PJ (2013)
Visual prostheses for the blind. Trends Biotechnol 31:562–571.
Shmiel T, Drori R, Shmiel O, Ben-Shaul Y, Nadasdy Z, Shemesh M, Teicher
M, Abeles M (2006) Temporally precise cortical firing patterns are associ-
ated with distinct action segments. J Neurophysiol 96:2645–2652.
Sminchisescu C, Kanaujia A, Metaxas D (2006) Conditional models for contex-
tual human motion recognition. Comput Vis Image Underst 104:210–220.
Stetter O, Battaglia D, Soriano J, Geisel T (2012) Model-free reconstruction
of excitatory neuronal connectivity from calcium imaging signals. PLoS
Comput Biol 8:e1002653.
Stopfer M, Jayaraman V, Laurent G (2003) Intensity versus identity coding in
an olfactory system. Neuron 39:991–1004.
Supekar K, Menon V, Rubin D, Musen M, Greicius MD (2008) Network
analysis of intrinsic functional brain connectivity in Alzheimer’s disease.
PLoS Comput Biol 4:e1000100.
Sutton C, McCallum A (2012) An introduction to conditional random fields.
FNT in Machine Learning 4:267–373.
Tang K, Ruozzi N, Belanger D, Jebara T (2016) Bethe learning of graphical
models via MAP decoding. InArtificial Intelligence and Statistics, pp
1096–1104. PMLR.
van den Heuvel MP, Stam CJ, Boersma M, Hulshoff Pol HE (2008) Small-
world and scale-free organization of voxel-based resting-state functional
connectivity in the human brain. NeuroImage 43:528–539.
Wang J, Zuo X, He Y (2010) Graph-based network analysis of resting-state
functional MRI. Front Syst Neurosci 4:16.
Yatsenko D, Josic K, Ecker AS, Froudarakis E, Cotton RJ, Tolias AS (2015)
Improved estimation and interpretation of correlations in neural circuits.
PLoS Comput Biol 11:e1004083.
Yoshimura Y, Dantzker JL, Callaway EM (2005) Excitatory cortical neurons
form fine-scale functional networks. Nature 433:868–873.
Yu S, Huang D, Singer W, Nikolic D (2008) A small world of neuronal syn-
chrony. Cereb Cortex 18:2891–2901.
Zuo XN, Ehmke R, Mennes M, Imperati D, Castellanos FX, Sporns O,
Milham MP (2012) Network centrality in the human functional connec-
tome. Cereb Cortex 22:1862–1875.