MSc Project
Marco Brigham
T
H
E
U
N
I V
E
R
S
I
T
Y
O
F
E
D
I
N B
U
R
G
H
Master of Science
Artiﬁcial Intelligence
School of Informatics
University of Edinburgh
2009
Abstract
A review of the ChialvoBak model is presented, for the twolayer neural network topology. A
novel Markov Chain representation is proposed that yields several important analytical quanti
ties and supports a learning convergence argument. The power lawregime is reexamined under
this new representation and is found to be limited to learning under small mapping changes.
A parallel between the power law regime and the biological neural avalanches is proposed. A
mechanism to avoid the permanent tagging of synaptic weights of the selective punishment rule
is proposed.
i
Acknowledgements
I wish to thank Dr. Mark van Rossum for his tireless support and attentive guidance, and for
having accepted to supervise me in the ﬁrst place.
To Dr. J. Michael Herrmann I wish to thank the very creative and rewarding discussions on the
holistic merits of the ChialvoBak model.
To Dr. Wolfgang Maass and his team at the Institute for Theoretical Computer Science at
T.U. Graz, I wish to thank the precious feedback and fruitful discussions received after the ﬁrst
talk on this MSc project.
ii
Declaration
I declare that this thesis was composed by myself, that the work contained herein is my own
except where explicitly stated otherwise in the text, and that this work has not been submitted
for any other degree or professional qualiﬁcation except as speciﬁed.
(Marco Brigham)
iii
To the memory of Per Bak, whose ideas live on and inspire.
iv
Contents
1 Introduction 2
1.1 Brief literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 The TwoLayer Topology 7
2.1 Basic Principles and Learning . . . . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Interference events . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.2 Synaptic Landscape . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.1.3 Neural avalanches . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.2 Storing Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2.2 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.3 Advanced Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.3.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.3.2 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3 Research Results 21
3.1 cband Saturation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.1.1 Desaturation strategies . . . . . . . . . . . . . . . . . . . . . . . 22
3.1.2 Global tag threshold . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.1.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2 Markov Chain Representation . . . . . . . . . . . . . . . . . . . . . . . . 26
3.2.1 Statistical properties . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.2.2 Markov chain representation: numerical evidence . . . . . . . . . 30
3.2.3 Analytical solution for Γ(2, j
n
, j
c
) . . . . . . . . . . . . . . . . . 33
3.2.4 Alternate formulation: graph transitions . . . . . . . . . . . . . . 36
3.2.5 Analytical solution: numerical evidence . . . . . . . . . . . . . . 37
3.2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.2.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.3 Learning Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.4 PowerLaw Behaviour and Neural Avalanches . . . . . . . . . . . . . . . 44
3.4.1 Biological interpretation . . . . . . . . . . . . . . . . . . . . . . . 45
3.4.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4 Conclusion 47
1
Chapter 1
Introduction
The ChialvoBak model was introduced by P. Bak and D. Chialvo [8] in 1999, with
the stated goal of identifying ”some ”universal” and ”simple” mechanism which allows
a large number of neurons to connect autonomously in a way that helps the organism
to survive” [8]. Their eﬀort resulted in a ”schematic ”brain” model of selforganised
learning and adaptation that operates using the principle of satisﬁcing” [1].
In common with other models authored by P. Bak, is a patent minimalism of form,
where models are succinctly deﬁned by simple, local and stochastic interaction rules
that reﬂect the most basic assumptions of the realworld system. However simple and
minimalistic, these models manage to reproduce complex and emergent behaviour that
is observed in the realworld systems [2].
In the ChialvoBak model, the basic properties of neurons and neural networks are
represented by simple, local and stochastic dynamical rules that support the processes
of learning, memory and adaptation. The most basic operations in the model, the
node activation and the synaptic plasticity rules, are regulated by ”WinnerTakeAll”
(WTA) dynamics and learning by synaptic depression, respectively. These mechanisms
may correspond to well accepted physiological mechanisms [14] [10] [12], which suggests
the biological feasibility of the model.
The present work focused on extending the analytical understanding of the model.
A Markov chain representation for the simple twolayer topology is proposed, where
the states of the chain correspond to the learning states of the network. This represen
tation provides a good statistical description of the model and supports an argument
for the learning convergence.
A power law tail in the learning time distribution, corresponding to an orderdisorder
phase transition in the model, was proposed by J.R Wakeling [20]. This result was
speciﬁc to the slow change mode, where the network is made to learn a succession of
small mapping changes. The power law behaviour was reexamined under the Markov
chain representation for other mapping change modes and was only reproduced in the
slow change mode.
An argument is provided for drawing a parallel between the above power law behaviour
and the biological neural avalanches, evidenced experimentally J. Beggs and D. Plenz
[4][5] in 2003. These correspond to the propagation of spontaneous activity in neural
2
networks with power law behaviour in the event size distribution.
The ability to store previously successful conﬁgurations is enabled by a selective pun
ishment mechanism [8] [1], where successful synaptic weights are depressed less severely
when no longer leading to the correct mappings. The selective punishment mechanism
has a known ageing eﬀect [1] that is related to the permanent tagging of the successful
synaptic weights. A mechanism to avoid the permanent tagging in order to maintain
the performance advantage of selective punishment is proposed.
This document is organised as follows:
Chapter 1 presents a succinct introduction to the ChialvoBak model and the rele
vant literature.
Chapter 2 introduces the ChialvoBak model in the simple twolayer topology, cover
ing the learning modes, the selective punishment rule and the powerlaw tail behaviour.
Chapter 3 presents the research results of this MSc project.
Chapter 4 presents the conclusion and future work.
1.1 Brief literature review
A brief review on the research papers related to the ChialvoBak model is presented be
low. The purpose of this review is to broadly describe the areas of the model that have
already been investigated to considerable depth. Detailed descriptions of the model
that support the present work are provided in Section 2.1.
The literature on the ChialvoBak model can be grouped into papers that follow the
original formulation of the model and papers that extend the model to diﬀerent work
ing principles and dynamic rules. As the present work is closely aligned to the ﬁrst
approach, so is the focus of the literature review presented below.
Papers on the original ChialvoBak model
The ChialvoBak model was introduced by P. Bak and D. Chialvo [8] in 1999. In this
paper, the motivations, biological constraints and ground rules are put forward, and
a great emphasis is placed on the biological plausibility of the model, which leads to
requirements of selforganisation at diﬀerent levels and robustness to noise.
Selforganised learning and adaptation is required to reﬂect ability to learn without
external guidance. The apparent lack of information in the DNA to encode the phys
ical properties of neurons and synapses and their connectivity [15] motivates the self
organisation at the connectivity level. Each neuron must learn without genetic or
external guidance, to which other neurons to connect and this connectivity should re
main ﬂexible in order to adapt to external changes.
The ability quickly recover from perturbations induced by biological noise is a con
straint motivated by the biological reality of the organism.
3
Learning by synaptic depression (negative feedback) is proposed as the basis of bi
ological learning and adaptation, supported by the following elements:
∙ Longterm synaptic depression (LTD) is as common in the mammalian brain as
longterm synaptic potentiation (LTP) [8]. The LTP mechanism is the suggested
physiological implementation of learning by synaptic depression.
∙ Learning by synaptic potentiation leads to very stable synaptic landscapes, from
which adaptation to new conﬁgurations is diﬃcult and slow.
∙ Learning of new tasks or adapting to new environments is error prone, as such, a
process that acts on errors rather than on what is correct leads to faster learning.
The other pillar of the model, the ”WinnerTakeAll” (WTA) rule is inspired from
models of SelfOrganised Criticality [2], as means to drive the system to an adaptive
critical state, where small perturbations can cause very large changes in the synaptic
landscape. The WTA rule plays also a key role in the solution to the credit assignment
problem by keeping the activity of the network low, as detailed in Section 2.1.
The synaptic plasticity changes are driven by a global signal informing on the success
of the latest synaptic changes. The ability for the organism to diﬀerentiate between
outcomes is deemed innate to the system and possibly resulting from Darwinist selec
tion.
A second paper from the same authors [1] was published in 2000 that expanded on
several key aspects of the model, such as the network topologies, the memory mech
anism and a new learning rule to tackle more complex problems. The performance
scaling under the new learning rule was also analysed.
Several network topologies and their relevant learning rules were formally deﬁned and
these are illustrated in Figure 1.1.
(a) The simple layered network
topology, which is the one used for
the present work.
(b) The lattice network
topology, where nodes
connect to a small
number of nodes in the
subsequent layer.
(c) The random network topology,
where nodes connect randomly to
n
of other neurons. Two subsets
of nodes are selected as the input
and the output nodes of the net
work.
Figure 1.1 – The various network topologies proposed in the original model. Figures
reproduced from [1].
4
The ability to store and retrieve previously successful conﬁgurations is enabled by the
selective punishment of synaptic weights, where previously successful weights are de
pressed less severely when no longer leading to the correct mappings.
A small modiﬁcation to the basic learning rules enables the network learn nonlinear
problems such as the XOR problem or more generally the generalised parity problem,
where the parity of any number of input neurons must be correctly calculated.
The learning of multistep sequences is introduced, where the depression of weights
related to bad sequences only occurred at the end of the last step. The generalisation
and feature detection capability is also covered. The network is able to diﬀerentiate
between classes of inputs requiring the same output by identifying useful features in
the inputs.
In [20], J.R. Wakeling identiﬁed an orderdisorder phase transition in the model that
is regulated by the ratio of middle layer to input and output nodes. At the phase
transition, the network displays powerlaw behaviour with exponent c = −1.3 for the
learning time distribution.
These orderdisorder regimes are characterised by the frequency of path interference
events, where already learnt mappings are accidentally destroyed while learning new
mappings. The disordered phase is characterised by a high probability of interference.
In [21], J. Wakeling investigated the performance of synaptic potentiation and selective
punishment under diﬀerent mapping change modes. In the slow change mode the net
work is made to learn a succession of mapping sets that only diﬀer by one inputoutput
mapping. In the )ij −)oj mode two diﬀerent mapping sets are presented alternately
to the network.
Synaptic potentiation was introduced to the plasticity rules by rewarding successful
weights by an amount i while still punishing unsuccessful weights. A quantitative
analysis showed that any small amount of synaptic potentiation resulted in higher av
erage learning time, specially in the slow change mode. This is illustrated in Figure
1.2a.
The performance of selective punishment was also investigated. While no visible im
provement was detected in the slow change mode, in the ﬂipﬂop mode the mechanism
was very eﬀective, as illustrated in Figure 1.2b.
ChialvoBak model extensions
In [7], R.J.C. Bosman, W.A. van Leeuwen and B. Wemmenhove propose an extension
to the ChialvoBak model by including the potentiation of successful weights. This
enables faster single mapping learning and multinode ﬁring in each layer, but is at the
cost of the adaptation performance.
In [16], K. Klemm, S. Bornholdt and H.G. Schuster propose a stochastic approximation
to the WTA rule and include a ”forgiveness” parameter in order to only punish those
weights that are consistently unsuccessful. This extension results in a more complex
model that according to the learning performance analysis in [1] does not improve on
5
(a) The learning performance of synaptic po
tentiation in the slow change mode, where only
one mapping is changed at a time.
(b) The learning performance of synaptic po
tentiation in the ﬂipﬂop mode, where the net
work is presented alternately with two diﬀerent
mappings.
Figure 1.2 – The learning performance decreases with any amount of synaptic potentiation
for the successful weights. Reproduced from [21].
the original model.
6
Chapter 2
The TwoLayer Topology
This chapter describes the functioning and the properties of the ChialvoBak model
that are most relevant to the research results presented in Chapter 3. As such, it does
not comprise an extensive description to the model, for which the reader is best directed
to the original papers by P. Bak and D. Chialvo [8] [1].
The contents of this chapter are based in the above two papers and in the paper
published by J.R. Wakeling [20] on the orderdisorder phase transition of the model.
2.1 Basic Principles and Learning
The ChialvoBak model is characterised by the following principles:
∙ ”WinnerTakeAll” dynamics: Neural activity only propagates through the
strongest synapses.
∙ Learning by synaptic depression (negative feedback): Synaptic plasticity
is exclusively applied by the weakening of synaptic weights that participate in
wrong decisions. These synapses are depressed.
These principles deﬁne the node activation and plasticity rules of the network, there
fore determining the dynamics and properties of the model. To illustrate this, the
functioning of a ChialvoBak network while learning an arbitrary inputoutput map
ping is presented below.
Consider a neural network with one input layer, one middle layer and one output
layer, as illustrated in Figure 2.1. Each layer has j
i
, j
n
and j
c
nodes respectively and
the network is noted Γ(j
i
, j
n
, j
c
).
The nodes connect between layers with synaptic weights n, as follows:
∙ Input nodes i connect to all middle nodes : with weights n(:, i).
∙ Middle nodes : connect to all output nodes o with weights n(o, :).
The network is initialised with random weights in [0, 1].
Each node can be active or not active, corresponding to node state 0 or 1. The activation
of an input layer node results in the activation of one middle layer node and one output
layer node, according to the following ”WinnerWakeAll” (WTA) rule:
7
Figure 2.1 – The twolayer network with three input nodes, four middle layer nodes and
three output nodes, i.e. j
= 3, j
= 4 and j
= 3. Each input node is connected to
all nodes in the middle layer, and each middle layer node is connected to all nodes in the
output layer.
∙ Input node i activates the middle layer node : with maximum n(:, i).
∙ Middle node : activates the output node o with maximum n(o, :).
In biological terms, the WTA rule could be implemented using lateral inhibitory con
nections within the same layer and excitatory connections between layers.
The activation sequence above ensures that no directed cycles are possible between
nodes, qualifying the network as a feedforward network.
For a given a set of weights, the WTA rule determines the sequence of activation in
the middle and output layers, which deﬁnes the active conﬁguration C of the network.
An active conﬁguration example is shown in Figure 2.2, where the blue connections
represent the winning weights according to the WTA rule.
Figure 2.2 – The ”WinnerTakeAll” rule (WTA) speciﬁes the active conﬁguration C of
the network. In the above graph all input nodes are shown active, whereas in the network
only one input node is active at the time. In this example C = {{1, 1, 1}, {2, 2, 3}, {3, 4, 2}}
and corresponds to the inputoutput mapping set ` = {1, 3, 2}. The blue connections
represent the active weights of the conﬁguration C.
An active conﬁguration C associates input nodes to middle layer nodes, which in turn
are associated to output nodes. As such, each C maps the input nodes to output nodes:
for each input node i corresponds a mapping to the output node o = `(i).
8
The mapping set ` contains the mappings of all the input nodes of the network.
In such terms, learning an arbitrary mapping set ` corresponds to the evolution of
synaptic weights from an initial active conﬁguration C to the ﬁnal active conﬁguration
˜
C that yields the required mapping set `.
The network learns an arbitrary mapping set ` by applying the following synaptic
plasticity rules:
1. A random input node i is selected.
2. The input node i ﬁres and activates a middle layer node : and output layer node
o according to the WTA rule.
3. If output node o is correct, i.e. o = `(i), return to step 1.
4. Otherwise depress the active weights n(:, i) and n(o, :) by a random amount
in [0, 1] and return to step 2.
A sequence of learning steps from ` = {1, 3, 2} to the identity mapping set
˜
` =
{1, 2, 3} is illustrated in Figure 2.3.
Figure 2.3 – The learning of the identity mapping set
˜
` = {1, 2, 3} from the initial map
ping set ` = {1, 3, 2}. In this example,
˜
` is learnt in three depressions: one depression
to learn input node 2 (upper row graph sequence) and two depressions to learn input node
3 (lower row graph sequence). The blue connections represent the active weights of the
conﬁguration and the orange connections represent the depression of active weights.
A weight normalisation can be applied at the end of step 3 of the plasticity rules, by
raising the weights of input node i and middle layer node : such that the winning
weights are equal to one.
9
The step 2 of the plasticity rules requires a feedback signal informing the suitabil
ity of the recent changes. This is provided in the form of a global feedback signal that
is broadcast to the entire network, in the case the latest changes are not satisfactory.
The synaptic plasticity rules could correspond to the following events at the biolog
ical level:
1. Depressing the current active level of an input node results in a new active level
that is tagged for recent changes. This tagging takes the form of a chemical or
hormone release that is triggered by the latest synaptic activity.
2. No further plasticity changes take place until a global feedback signal is received.
This signal is broadcast to the entire network informing whether the latest changes
are not satisfactory.
3. Following an unsuccessful global feedback signal, the step 1 is repeated. No
further actions are taken otherwise.
The synaptic plasticity rules result in the following properties:
∙ For the global feedback mechanism to be eﬃcient in directing synaptic learning,
the rate of plasticity change has to be sparse. In such conditions, the credit
assignment problem [18][3] is solved, i.e. the system can determine which elements
are to be punished following bad performance.
∙ The network signalling is in the time scale of ﬁring patterns (i.e. milliseconds),
while the tagging and feedback mechanisms are in a timescale more adapted to
the scale of events in the external world (i.e. seconds to hours).
∙ The global feedback signal represents an external critic rather than a teacher, as
no speciﬁc instructions are provided to direct the plasticity activity.
For the network to learn a random mapping set ¹, the middle layer size must be at
least be as large as the input layer size, i.e. j
n
≥ j
i
, so that each input node i can
have a dedicated path to the corresponding output node o = `(i).
A network with j
i
input nodes, j
n
middle layer nodes and j
c
output nodes is noted
Γ(j
i
, j
n
, j
c
).
2.1.1 Interference events
In the process of learning a new mapping an interference event i may occur, where the
network unlearns a previously learnt mapping.
This is the case whenever while learning a new mapping, a middle layer node that
was establishing a correct mapping for another input node is selected (assuming the
correct output node for these input nodes is diﬀerent). This is illustrated in Figure 2.4.
2.1.2 Synaptic Landscape
An interesting consequence of learning by synaptic depression is the resulting synaptic
landscape, shown in Figure 2.5.
10
Figure 2.4 – While learning a mapping the network may unlearn a previously correct
mapping. In the above sequence, the learning of input node 3 lead from ` = {1, 2, 2}
to
˜
` = {1, 1, 3}. As such, the net number of learnt mappings remained unchanged: the
output mapping of input node 3 was learnt and the output mapping of input node 2 was
unlearnt.
20 40 60 80 100
0
0.2
0.4
0.6
0.8
1
Node 1 to Middle Layer
Γ(6,108,6) [runs:1e+5 rand]
W
e
i
g
h
t
w
Node Index
(a) The synaptic weights from input node one
to the middle layer, in a network Γ(6, 108, 6).
0 0.2 0.4 0.6 0.8 1
0
0.005
0.01
0.015
0.02
0.025
Weight Distribution p(w)
Γ(32,1024,32) [runs:1e+5 bins:100 rand]
p
(
w
)
Weight w
(b) The synaptic weights distribution (100
bins) in a network Γ(6, 108, 6).
Figure 2.5 – The metastable synaptic landscape is a direct consequence of learning by
synaptic depression and supports the fast adaptation property of the model.
In Figure 2.5a the metastable nature of the synaptic landscape is apparent, with the
active conﬁguration barely supporting the current mapping. This is to be contrasted
with the synaptic landscape resulting from learning by synaptic potentiation, which
often results in a small number of dominating synaptic weights.
In this model, learning a very diﬀerent mapping set ` is often just a few depres
sions away from the currently active weights.
The particular form of the weights distribution in Figure 2.5b is due to both the WTA
rule and learning by synaptic depression. As the active synapses for a given input or
middle layer node are depressed by a random amount, the WTA rule will select the
synapse with the current highest weight
1
for the new active conﬁguration in each layer.
This amounts to shifting all the weights by the amount of the diﬀerence between the
previous highest weight and the new highest weight.
1
The highest weight after depression may still correspond be the previous winning weight, but the
probability of reselection is lower than for any another weight.
11
Starting from an uniform weights distribution and repeating the above process a suf
ﬁcient number of times, yields the distribution in Figure 2.5b. The intermediate steps
of this process are illustrated in Figure 2.6.
0 0.2 0.4 0.6 0.8 1
0
0.005
0.01
0.015
0.02
0.025
Weight Distribution p(w)
Γ(32,1024,32) [runs:8 bins:100 rand]
p
(
w
)
Weight w
(a) The synaptic weights distribution after
adapting to eight successive random mappings,
in a network Γ(32, 1024, 32).
0 0.2 0.4 0.6 0.8 1
0
0.005
0.01
0.015
0.02
0.025
Weight Distribution p(w)
Γ(32,1024,32) [runs:15 bins:100 rand]
p
(
w
)
Weight w
(b) The synaptic weights distribution after
adapting to 15 random mappings, in a network
Γ(32, 1024, 32).
Figure 2.6 – The synaptic weights distribution evolves from a uniform distribution at the
initialisation of the network, to the distribution in Figure 2.5b.
2.1.3 Neural avalanches
The learning performance of the network can be measured in the number of depressions
j required to completely learn a given mapping set `. This quantity will be loosely
referred to as the learning time, although no particular timescale is thereby implied.
The learning performance is known [8] to improve with increasing middle layer sizes,
as illustrated in Figure 2.7. This is an advantage to regular backpropagation learning
where in general the performance decreases with increasing middle layer size.
Let A be the random variable associated with the number of depressions j required to
learn a mapping ` and let 1:(A = j) ≡ 1:(j) be the probability of learning mapping
` in j depressions.
The learning performance of the network Γ(j
i
, j
n
, j
c
) is completely determined by
the learning time distribution, characterised by the probability mass function j such
that
j(j) = 1:(A = j) (2.1)
∞
∑
j=0
j(j) = 1. (2.2)
The basic operation for measuring j(j) is to record the number of depressions j re
quired to learn the current mapping set `, increase by one unit the count of mapping
set learning in j depressions, present a new mapping set
˜
` to the network and so on.
However, certain aspects in the setup of the simulations have a noticeable impact on
the measured values, as such these will be discussed in greater detail below.
12
20 40 60 80 100
0
50
100
150
200
250
Average Learning Time 〈ρ〉
Γ(6,*,7) [run:50*1e+4 slow]
Middle Layer Nodes
〈
ρ
〉
(a) The average number of depressions ⟨j⟩ re
quired to learn a random mapping A for diﬀer
ent sizes of the middle layer.
20 40 60 80 100
0
5
10
15
20
25
30
Average Interference Events 〈ν〉
Γ(6,*,7) [run:50*1e+4 slow]
Middle Layer Nodes
〈
ν
〉
(b) The average number of interference events
⟨i⟩ while learning a random mapping A for
diﬀerent sizes of the middle layer.
Figure 2.7 – Increasing the number of middle layer nodes decreases the average learning
time ⟨j⟩ and the average interference events ⟨i⟩. The plots show ⟨j⟩ and ⟨i⟩ for networks
with six input nodes, seven output nodes and varying number of middle layer nodes.
One could require the weights of the network to be reset prior to learning the next
mapping but this would lead to measuring the ﬁrst mapping learning times. Instead,
the weights of the network are not reset at each new mapping set, and this yields a
measure that is closer to online learning performance.
A further distinction can be made on the degree of similarity between the new mapping
set being presented to the network and the previous one. These can be completely ran
dom or diﬀering in a small number of mappings only. Borrowing from J.R. Wakeling
[20], the slow mapping change mode corresponds to one single mapping change
2
in the
new mapping set. The random mapping change mode corresponds to random mapping
sets being presented to the network.
The distribution j(j) for a network Γ(8, ∗, 9) is shown in Figure 2.8a. The tail of
the distribution (i.e. long learning times) recedes for larger middle layer sizes, which is
consistent with the plot in Figure 2.7.
An interesting aspect of the model is the power law tail of j(j) [20], as illustrated
in Figure 2.8b. The power law tail is a telltale sign of scalefree behaviour, for which
no single value of j is typical for the learning time in those networks. Shorter values
of j occur more frequently than longer ones, but the later occur frequently enough to
not being singled out as exceptional.
The powerlaw tail of j(j) can be understood as avalanches of activation in the middle
layer nodes. Borrowing terminology from statistical physics, three operating regimes
are then identiﬁed:
∙ Subcritical regime for j
n
<< j
i
j
c
2
The slow change learning mode requires j
≥ j
+ 1 unless the input nodes can share the same
output node.
13
10
1
10
2
10
3
10
4
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
[runs:1e+6 slow]
p
(
ρ
)
ρ
Γ(8,36,9)
Γ(8,72,9)
Γ(8,144,9)
(a) The learning time distributions for several
middle layer sizes of a network with eight input
and eight output nodes. Data from 1c +6 runs.
10
1
10
2
10
3
10
4
10
−6
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
[runs:1e+6 slow]
p
(
ρ
)
ρ
Γ(8,72,9)
Γ(16,272,17)
Γ(32,1056,33)
Γ(64,4160,65)
(b) The learning time distributions for several
networks with critical number of middle layer
size. Data from 1c + 6 runs.
Figure 2.8 – The learning time distributions reveal three distinct regimes: subcritical,
critical and supercritical. The critical regime exhibits power law behaviour with j(j) ∼
j
−1.3
according to [20].
∙ Critical regime for j
n
∼ j
i
j
c
∙ Supercritical regime for j
n
j
i
j
c
In [20], J.R. Wakeling proposed that the power law tail of j(j) corresponds to an
orderdisorder phase transition in the model and that the key diﬀerence in the learning
dynamics for the three operating regimes is the interference probability 1:(i):
∙ In subcritical networks, there are enough middle layer nodes for interference
events to be quite rare and therefore learning is very quick.
∙ In supercritical networks, there are hardly enough middle layer nodes to learn
without inducing interference events and therefore learning is extremely slow.
∙ The learning dynamics for critical network sizes is inbetween the other two
regimes with just enough interference to occasionally cause large learning times
while most of the time the learning times are quite fast.
However, it should be noted that the model has not been proved to be critical in the
proper statistical physics sense, in order to merit such terminology. Assessing the crit
icality for the model in the twolayer topology is certainly challenging.
Furthermore, the approximately straight segments in the distributions of Figure 2.8b do
not necessarily imply that j(j) is a proper power law tail distribution, as very clearly
explained in the paper [9] by Clauset, Shalizi and Newman. Straight segments in a
loglog plot are a necessary but not suﬃcient condition for j(j) to be a power law tail
distribution. Due to timing constraints however, no conclusive power law testing was
completed for j(j) and in consequence, the terminology proposed in [20] is adopted
through the document.
2.1.4 Summary
The key elements of this section are the following:
14
∙ Network dynamics: Deﬁned by ”WinnerTakesAll” dynamics and learning by
synaptic depression (negative feedback).
∙ Inputoutput learning: The network is able to learn an arbitrary mapping set
` where for each input node i corresponds an output node o = `(i).
∙ Local ﬂagging mechanism: Plasticity changes are locally marked for recent
activity.
∙ Global feedback mechanism: Feedback is provided in the form of a global
feedback signal specifying whether the most recent changes are unsatisfactory.
∙ Solution to the creditassignment problem: Requires sparse network activ
ity, so that no plasticity changes occur until a global feedback signal is received.
∙ Two typical timescales: The network signalling occurs in the time scale of
the ﬁring patterns, while the tagging and feedback mechanisms occur in a much
longer timescale that is relevant to the scale of events in the external world.
∙ Interference event i: The learning of inputoutput mappings can be disrupted
by the unlearning of previously learnt mappings.
∙ Metastable synaptic landscape: The active conﬁguration is barely supported
by the winning weights.
∙ Neural Avalanches: For middle layer sizes j
n
= j
i
j
c
the network displays
powerlaw behaviour in the distribution of the learning time j(j).
2.2 Storing Mappings
The plasticity rules introduced in Section 2.1 enable the network to learn a random
mapping set `, and quickly adapt to another mapping set
˜
` whenever needed. Not
much information is left [1] in the synaptic weights to reliably retrieve ` at a later
stage, since active weights that supported ` were depressed
3
a random amount in [0, 1]
to support the new mapping set
˜
`.
An additional mechanism is therefore required to store the information from previ
ously learnt mapping sets for later recall. It turns out that such a mechanism exists
and amounts to depressing less severely the weights that have successful in the past,
and is called the selective punishment rule [8][1].
The selective punishment rule requires small modiﬁcations to the plasticity rules to
enable the distinction between successful and unsuccessful weights:
1. A random input node i is selected.
2. Input node i ﬁres and activates a middle layer node : and output node o according
to the WTA rule.
3. If output node o is correct, i.e. o = `(i), tag the weights n(:, i) and n(o, :) as
successful and return to step 1.
3
More speciﬁcally, the active weights that are not shared by A and
˜
A.
15
4. Otherwise depress the active weights n(:, i) and n(o, :) by:
∙ A random amount in [0, 1] if n(:, i) and n(o, :) have never been successful.
∙ A random amount in [0, c] otherwise.
Return to step 1.
In the ChialvoBak model, recalling a mapping set ` refers to a diﬀerent operation
than in other neural network models. Since synaptic plasticity is required to retrieve
the information stored in the synaptic weights, the network is rather readapting to a
previously seen mapping set, than recalling the mapping set. Nevertheless, in order to
distinguish from the learning rules without selective plasticity, the term recall will be
used.
0 10 20 30 40 50
0
100
200
300
400
500
Recall Time ρ
Γ(6,36,6) [runs:50 rand]
ρ
Recall
M
1
M
2
M
3
M
4
(a) Example of learn and recall performance
without selective punishment.
0 10 20 30 40 50
0
100
200
300
400
500
Recall Time ρ
Γ(6,36,6) [runs:50 select rand]
ρ
Recall
M
1
M
2
M
3
M
4
(b) Example of learn and recall performance
with the selective punishment rule.
Figure 2.9 – The number of depressions j required to ﬁrst learn and then recall four
random mapping sets `
1
, ⋅ ⋅ ⋅ , `
4
. The network is presented with the mapping in random
succession, and value of j is recorded for each graph, i.e. at recall = 10 the network has
seen each mapping set 10 times.
The selective punishment rule results in a dramatic performance increase (under the
random mapping change mode), as shown in Figure 2.9. This performance increase
results from the network establishing preferred paths from each input node to the out
put nodes required by the mappings being presented. These preferred paths are the
ﬁrst to be queried when the active conﬁguration is no longer correct. A detailed ex
ample of the selective punishment dynamics is presented in the Appendix of this section.
The weights tagged by the selective punishment rule are constrained to a region
4
within
a distance c from unity, as shown in Figure 2.10. This region is referred to as the cband.
2.2.1 Summary
The key elements of this section are the following:
4
The uniform distribution of weights in the c − /ond in Figure 2.10 results from depressing the
tagged weights a ﬁxed amount c rather than a random amount in [0, c]. In the later case, the resulting
distribution of weights in the cband would be similar to that of Figure 2.5b.
16
0.998 0.9985 0.999 0.9995 1
0
0.005
0.01
0.015
0.02
0.025
Weight Distribution p(w)
Γ(32,1024,32) [runs:1e+5 bins:5e+4 select rand]
p
(
w
)
Weight w
Figure 2.10 – The weights tagged by the selective punishment rule are kept within the c
band located in [1 − c, 1]. This is where the memory of previous successful mappings is
stored. Discrete distribution of 500 bins.
∙ Selective punishment rule: Recalling previously learnt mapping sets is enabled
by depressing weights that haven been successful in the past by a smaller random
amount c when no longer leading to the desired mapping.
∙ Selective punishment dynamics: The performance increase results from the
network establishing preferred paths from input nodes output nodes as required
by the learnt mapping sets. On average these preferred paths are queried much
more often.
∙ Delta band: contains the weights representing the memory of previously suc
cessful mappings and is located at [1 −c, 1].
2.2.2 Appendix
The example from Figure 2.9 will be used to illustrate the dynamics of selective pun
ishment. Suppose that `
1
, ⋅ ⋅ ⋅ , `
4
are presented to the network in that order and
require input one to activate output nodes {1, 3, 6, 6} respectively.
After learning the mapping from input one to output node required by `
1
, the winning
weights n(:
1
, 1) and n(1, :
1
) are tagged by the selective punishment rule.
When presented with `
2
, input node one should now activate output node three and
the weights n(:
1
, 1) and n(1, :
1
) are depressed accordingly. This results in a series
of successive depressions to bring these weights slightly below the respective second
highest weights.
This succession of depressions is necessary since the average weight distance for this
network is 1,j
n
∼ 0.03 for the weights set n(:, i) and 1,j
c
∼ 0.14 for the weights set
n(o, :), whereas n(:
1
, 1) and n(1, :
1
) are now depressed by small random amounts
in c ≤ 0.001. This accounts for the relatively higher adaptation values for learning
mappings `
2
, `
3
and `
4
in Figure 2.9b, when compared to Figure 2.9a.
This also illustrates the negative performance impact that synaptic potentiation would
have in this network, since it would lead to large weight diﬀerences (in units of depres
sion amounts) between the active weights and the other weights. A metastable synaptic
17
landscape, such as the one illustrated in Figure 2.5a, is a requirement for the network
to quickly converge to a new mapping sets.
Eventually, either n(:
1
, i) or n(1, :
1
) will be depressed below the other weights.
Supposing that n(:
1
, i) is ﬁrst, then node one switches from middle layer node :
1
to another middle layer node ˜ :, which has 1,j
c
probability of activating output node
three. If that is the case, the input node one has learnt the correct mapping for `
2
and the weights n( ˜ :, 1) and n(3, ˜ :) are also tagged as successful.
If middle layer node ˜ : does not lead to output node three, it is depressed accord
ingly. Node one is then very likely to switch back to middle layer node :
1
, which
will still activate output node one, unless weight n(1, :
1
) is already the second high
est weight of node one, where it then has a chance of activating a diﬀerent output node.
If output node three is still not found after a few more depressions, the search se
quence will now alternate between middle layer node :
1
and other middle layer nodes,
and the output node of middle layer node :
1
will alternate between output node one
and the other output nodes.
After learning the mapping sets `
1
, ⋅ ⋅ ⋅ , `
4
for the ﬁrst time, each input node has
one or more preferred paths formed by the pairs of tagged weights that lead to the
required output nodes. The network will pool these weights much more frequently.
The example above can be easily generalised to other input nodes and to the weight
n(1, :
1
) reaching the second highest weight before weight n(:
1
, 1) does.
A sample run where the mapping sets `
1
, ⋅ ⋅ ⋅ , `
4
required input one to activate out
put nodes {1, 3, 6, 6}, resulted in the preferred paths for input node shown in Table 2.1:
Preferred paths from input 1
To middle layer node : From middle layer node : to output node o
3 6
12 2, 6
30 6
31 1
34 3
Table 2.1 – The successful weights tagged by the selective punishment rule result in a set
of preferred paths for input node 1.
The preferred path to output node two, which is not required for input node one, was
added by input node two when learning mapping set `
4
.
2.3 Advanced Learning
The type of learning problems that the model can tackle so far are better described as
solving a routing problem: given a mapping set `, the input nodes i have to ﬁnd paths
to the output node `(i).
18
A more advanced type of learning consists in considering mapping sets between
input node activation patterns 1 and the speciﬁc activation of output nodes o = (1).
As before, the state of each input node can be active i = 1 or inactive i = 0 and the
entire conﬁguration of input nodes is represented by a binary vector 1. For example,
1 = {1, 0, 0, 1, 0, 1} is an activation pattern for a network with six input nodes.
Learning the basic Boolean functions = {AND, OR, XOR, NAND, ⋅ ⋅ ⋅ } is a par
ticular example of this type of learning. The logical value of proposition j and ¡ is
represented by the state of two input nodes and the logical value of the function (j, ¡)
is represented by the activation of one of the two output nodes.
The changes to the plasticity rules that are required to learn this type of problem
are surprisingly small and amount to a slightly modiﬁed ”WinnerTakeAll” rule:
∙ Input conﬁguration 1 = {r
1
, ⋅ ⋅ ⋅ , r
j
} activates the middle layer node , with
maximum ℎ
)
=
∑
j
i
n(,, i) r(i).
∙ Middle node : activates the output node o with maximum n(o, :), as before.
A bias input node that is always active is necessary in order to compute the state where
the remaining input nodes are inactive.
An example of the network solving the XOR problem under the above plasticity rules
is shown in Figure 2.11. An example of weights that implement a solution to the XOR
problem are shown in Table 2.2 of the Appendix of this section.
Figure 2.11 – An example active conﬁguration implementing the XOR truth table.
The network can learn the XOR problem with three middle layer nodes. In general, it
can learn any mapping with j
n
= 2
j
middle layer nodes, by discovering for each
input pattern 1 the corresponding middle layer node pointing to the correct output
(1). As there are as many middle layer nodes as input conﬁgurations the learning
convergence is guaranteed. As can be appreciated from the XOR example, the network
can learn with fewer middle layer nodes but the exact minimum is dependent on .
2.3.1 Summary
The key elements of this section are the following:
∙ Advanced learning capability: The model can learn the mapping of input
node conﬁgurations 1, representing the activation state of the input nodes, and
the respective output nodes o = (1). In particular the basic Boolean functions
can be learned.
19
∙ Advanced learning plasticity rule: The middle layer node with the maximum
weighted sum of weights n(:, i) is selected and activates the output node as
before.
2.3.2 Appendix
An example of weights that implement a solution to the XOR problem are shown in
Table 2.2.
Input to middle w(m,i) Middle to output w(o,m)
1 → 1 0.5 1 → 1 0.1
1 → 2 0.4 1 → 2 0.2
1 → 3 0.1
2 → 1 0.3 2 → 1 0.4
2 → 2 0.7 2 → 2 0.3
2 → 3 0.9
3 → 1 0.2 3 → 1 0.5
3 → 2 0.6 3 → 2 0.6
3 → 3 0.8
Table 2.2 – Example weights to solve the XOR problem under the advanced learning
plasticity rules.
20
Chapter 3
Research Results
This chapter presents the results of the research that was conducted during this MSc
project.
3.1 band Saturation
The selective punishment rule enables the network to quickly readapt to previously
learnt mappings by depressing less severely weights that were successful at least once
in the past. It is also know that the performance of this mechanism has an ageing eﬀect
at large time scales [1], as illustrated in Figure 3.1.
0 10 20 30 40 50
0
100
200
300
400
500
Recall Time ρ
Γ(6,36,6) [runs:50 select rand]
ρ
Recall
M
1
M
2
M
3
M
4
(a) At short time scales the selective punish
ment rule drastically improves the ability to
quickly recall previously learnt mapping sets.
0 500 1000 1500 2000
0
100
200
300
400
500
Recall Time ρ
Γ(6,36,6) [runs:2000 select rand]
ρ
Recall
M
1
M
2
M
3
M
4
(b) At long time scales the recall performance
degrades progressively.
Figure 3.1 – The ageing eﬀect on the selective punishment performance at large time
scales.
As before, the term recall will refer to the readaptation of the network to a previously
learnt mapping `.
The performance degradation of selective punishment is caused by the saturation of
the cband, which is the region within a distance dcto from unity where the weights
tagged as successful are constrained to be.
21
For the selective punishment rule to be eﬀective, each input node should be able to
quickly sort through the tagged weights to recover the preferred path to the correct
output node. Ideally, each input node would have established one preferred path for
each previously learnt output nodes. As such, it would take a number of depressions
in the order of the number of preferred paths to ﬁnd the correct output node.
On the other hand, if the number of preferred paths leading to the same output node
grows further, the advantage of the tagging mechanism to identify the preferred middle
layer nodes ceases to be eﬀective.
The increase of paths leading to the same output node is a consequence of all weights
eventually being given a chance to participate in correct mappings. Consider the con
tinuous raising of the weights caused by the depressions of the plasticity rules. Each
raising step is the diﬀerence between the highest weight and the second highest. Even
tually, all weights endup in the cband and are soon able to compete with the tagged
weights for participating in a correct conﬁguration. Once that occurs, one additional
path to the output node is created.
The monotonous increase of tagged weights leads to a saturation of the cband, as
increasingly more weights are conﬁned in that region of weight space. This eﬀect can
be appreciated in Figure 3.2a and the corresponding increase of recall times is illustrated
in Figure 3.2b.
0 50 100 150 200 250
0
0.2
0.4
0.6
0.8
1
δ−band Saturation
Γ(6,36,6) [runs:100x250 map:128 rand]
T
a
g
g
e
d
W
e
i
g
h
t
s
(
%
)
Recall
(a) The number of tagged weights increases
monotonically and leads to a saturation of the
cband.
0 50 100 150 200 250
0
50
100
150
Average Recall Time 〈ρ〉
Γ(6,36,6) [runs:100x250 map:128 rand]
〈
ρ
〉
Recall
not selective
selective
(b) The performance of the selective punish
ment rule degrades with successive recalls.
Figure 3.2 – As the percentage of weights tagged as correct increases, the recall times
approach the performance of the regime without selective punishment.
3.1.1 Desaturation strategies
The monotonous increase of tagged weights is a consequence of the permanent tagging
of the selective punishment rule. As such, a mechanism is required to reduce the tag
ging lifespan.
P. Bak and D. Chialvo proposed [1] a mechanism of neuron ageing to tackle this issue,
where nodes are replaced at a ﬁxed rate, their weights randomised and the tagging
22
information removed. However, the neuron replacement rate may be dependent on the
level of network activity, in order to successfully counter the saturation rate of the
dctoband.
Several strategies that result in nonpermanent tagging were reviewed and the ﬁrst
of them was selected for implementation:
∙ Global tag threshold: weights are untagged if not successful for more than a
global threshold of depressions.
∙ Local tag threshold: as the previous, but the threshold depends on the past
performance of each weight.
∙ Interference correction: untagging weights after a threshold number of inter
ference events.
3.1.2 Global tag threshold
The global tag threshold has been investigated in greater detail, and numerical simu
lations suggest that an optimum threshold value exists for each network size.
In order to ensure that the optimal tag threshold value does not depend on the activity
level of the network, an increasing number of mappings was presented to networks of
the same size. The performance of the optimal tag threshold value was consistent across
the number of presented mapping sets, as illustrated in Figure 3.3. For the network
Γ(6, 36, 6) the optimal tag threshold value is close to 48.
10
1
10
2
10
3
0
0.2
0.4
0.6
0.8
1
Average δ−band Saturation
Γ(6,36,6) [runs:50x250 rand]
A
v
e
r
a
g
e
T
a
g
g
e
d
W
e
i
g
h
t
s
(
%
)
Mappings
16
32
48
64
80
(a) The average saturation of the cband for
global tag threshold values are consistent across
the number of presented mapping sets.
10
1
10
2
10
3
0
50
100
150
200
Average Recall Time 〈ρ〉
Γ(6,36,6) [runs:50x250 rand]
〈
ρ
〉
Mappings
16
32
48
64
80
(b) The average recall time ⟨j⟩ performance for
global tag threshold values are consistent across
the number of presented mapping sets.
Figure 3.3 – For the network Γ(6, 36, 6) the optimal global tag threshold value is close to
48.
For networks with larger number of input nodes j
i
the optimal tag threshold value is
also higher. This was veriﬁed for the network Γ(12, 144, 12), where the optimal tag
threshold values is around 64. This is illustrated in Figure 3.4.
The optimal tag threshold seems closely related to an optimal number of average tagged
23
10
1
10
2
10
3
0
0.05
0.1
0.15
0.2
Average δ−band Saturation
Γ(12,144,12) [runs:50x250 rand]
A
v
e
r
a
g
e
T
a
g
g
e
d
W
e
i
g
h
t
s
(
%
)
Mappings
32
48
64
80
(a) The average saturation of the cband for
global tag threshold values.
10
1
10
2
10
3
50
100
150
200
250
300
Average Recall Time 〈ρ〉
Γ(12,144,12) [runs:50x250 rand]
〈
ρ
〉
Mappings
32
48
64
80
(b) The average recall time ⟨j⟩ performance for
global tag threshold values.
Figure 3.4 – For the network Γ(12, 144, 12) the optimal global tag threshold value is around
64.
middle layer nodes and tagged output nodes behind them, as illustrated in Figure 3.5.
Tag threshold values that are too low result in the network forgetting successful nodes
too fast, as can be observed by the sharp decrease in the average recall time ⟨j⟩ in
Figure 3.5. Tag threshold values that are too high fail to get rid of path redundancy
fast enough. This results in an increase of the average recall time ⟨j⟩, as illustrated in
Figure 3.4b for the tag threshold value of 80, for example. Somewhere inbetween lies
the optimal value.
The blue line in Figure 3.5b represents the average number of tagged middle layer
nodes per input node. From Table 3.1, the number of tagged middle layer nodes sur
passes the number of output nodes j
c
from tag threshold 32 and above.
The green line of the same graph represents the average number of tagged output
nodes per tagged middle layer node. This value decreases until reaching tag threshold
32, where the network compensates for the lack of enough tagged middle layer nodes,
by tagging more output nodes per tagged middle layer node. Near tag threshold 32 a
minimum is reached, and starts growing again as higher tag threshold values cannot
get rid of path redundancy fast enough.
The best tag threshold found for this network produced an average of 7.9 tagged middle
layer nodes, which is nearly two nodes more than the number of output nodes j
c
. The
full results are presented in Table 3.1 in the Appendix of this Section.
3.1.3 Summary
The key elements of this section are the following:
∙ cband saturation: The permanent tagging of successful weights results in the
monotonous increase of weights being constrained to the cband region, therefore
reducing the performance advantage of the selective punishment rule.
∙ Desaturation strategies: Several desaturation strategies are possible and amount
24
Global Threshold Average Middle Layer Average Output Layer
16 4.8350 1.5447
24 5.3833 1.5129
32 6.0450 1.4118
40 6.9167 1.4135
48 7.9133 1.5362
56 9.3767 1.6998
64 10.9283 1.9372
72 12.4167 2.1066
80 14.4617 2.3799
Table 3.1 – Average number of tagged middle layer nodes per input node and average
number of tagged output nodes per tagged middle layer node for diﬀerent values of the
global tag threshold in the network Γ(6, 36, 6).
10 20 30 40 50 60 70 80
0
50
100
150
200
Average Recall Time 〈ρ〉
Γ(6,36,6) [runs:100x250 map:128 rand]
〈
ρ
〉
Global Tag Threshold
(a) The average recall time ⟨j⟩ decreases
sharply until reaching the optimal global tag
threshold value and slowly starts increasing
again beyond that point.
10 20 30 40 50 60 70 80
0
5
10
15
20
25
Average Tagged Nodes (Middle−Output)
Γ(6,36,6) [runs:100x250 map:128 rand]
A
v
e
r
a
g
e
N
o
d
e
s
Global Tag Threshold
middle
output x10
(b) The average number of tagged middle
layer nodes steadily increases with increasing
global tag threshold values. The average num
ber of tagged output nodes per tagged middle
layer node has a minimum when the aver
age tagged middle layer nodes are equal to the
number of output nodes j
.
Figure 3.5 – The relation between the optimal global tag threshold and the average number
of tagged middle layer nodes and average number of tagged output nodes per tagged middle
layer nodes.
to capping the tagging lifetime.
∙ Global tag threshold: Sets a global limit on the number of times a tagged
weight can be wrong before becoming untagged. There is an optimal tag threshold
for each network size that is independent of the level of network activity.
∙ Global tag threshold dynamics: The optimal value of the global threshold is
related to the average number of tagged middle layer nodes and average number
of tagged output nodes behind them.
25
3.2 Markov Chain Representation
A ChialvoBak network can be represented by a ﬁrstorder Markov chain when consid
ering the evolution of the network as a sequence learnt mappings states. Such repre
sentation is useful to derive several statistical properties of the model analytically, such
as the average learning time ⟨j⟩, the learning time distribution j(j) and the average
interference events ⟨i⟩.
The Markov chain representation seems appropriate since the evolution of the net
work is to a large extent stochastic. The random amounts of the depression from the
plasticity rules result in changes to the active conﬁguration C, which is the basic macro
scopic state of the network. Assuming the transition between active conﬁguration C to
a new active conﬁguration
˜
C is stochastic and only depends on C, then the evolution
of the network can be described by a ﬁrstorder Markov chain
1
.
Rather than considering the evolution between active conﬁgurations C, a more mean
ingful basic state of the Markov chain representation is the learning state o
a
of the
network, i.e. the number n of currently learnt mappings. For each active conﬁguration
C the learning state o
a
is determined by counting the number n of learnt mappings
of C. As such, the correspondence between C and o
a
maintains the Markov chain
properties mentioned above.
The Markov chain has j
i
+ 1 states, noted o
0
, o
1
, ..., o
j
, corresponding to learning
from zero up to j
i
mappings.
In this context, learning a new mapping set ` corresponds to the Markov chain start
ing from an initial state o
i
and evolving towards the ﬁnal state o
j
. The chain has a
transition from o
a
to o
a+1
when an additional output node is learnt and from o
a
to
o
a−1
in the case of an interference events. This is illustrated in Figure 3.6.
Figure 3.6 – The Markov chain representation considers the evolution of the network in
terms of the number of learnt mappings. In this example, the network started in the state
o
2
and successfully reached the ﬁnal state o
3
after two depressions. The evolution sequence
was o
2
⇒ o
2
⇒ o
3
. The target mapping set is ` = {1, 2, 3}
The ﬁnal state o
j
is special since the Markov chain stops when arriving to state o
j
1
A Iorder Markov chain would depend on the I previous steps.
26
and no transitions from o
j
to any other states is possible. This corresponds to the
network having fully learnt the mapping set ` and therefore no further depressions
are necessary. The fact that all states can reach the absorbing state o
j
and that no
transition is possible from o
j
to any other state qualiﬁes the Markov chain as an ab
sorbing Markov chain.
In general, there are four possible transitions from a given state o
a
. Noting o(t) the
state of the system at evolution step t, if o(t) = o
a
then o(t+1) ∈ {o
a−1
, o
a
, o
a+1
, o
a+2
}.
In Figure 3.7 is shown an example of transition o
a
⇒ o
a+2
.
Figure 3.7 – The network can learn up to two mappings after one depression giving the
transitions o
⇒ o
+1
and o
⇒ o
+2
, and it can unlearn one single mapping giving
o
⇒ o
−1
.
For networks with three input nodes, all possible transitions are shown in Figure 3.8,
where the arrows indicate the sense of the transitions.
Figure 3.8 – All the possible transitions for a network with three input node is shown
above. The arrows indicate the sense of the transitions. No transition is possible from o
3
to any of the other states.
A Markov chain is completely determined by the state transition matrix A, which in
the columns speciﬁes the transition probabilities between states o
a
, and the initial state
probability vector p, which speciﬁes the initial state probabilities 1:(o(0)).
The element o
na
of the state transition matrix A is the probability 1:(o
n
∣o
a
) of
the transition o
a
⇒ o
n
. The element j
I
of the column vector p is the probability of
starting the chain in state 1:(o(0) = o
I−1
).
27
For the elements of A and p to represent valid probabilities the following must hold:
j
∑
n=0
o
na
= 1 for any column index n of A (3.1)
j
∑
I=0
j
I
= 1 (3.2)
For a network with three input nodes, A and p have the following form:
A =
⎛
⎜
⎜
⎝
o
00
o
01
0 0
o
10
o
11
o
12
0
o
20
o
21
o
22
0
0 o
31
o
32
1
⎞
⎟
⎟
⎠
and p = (j
0
j
1
j
2
j
3
)

,
where o
30
= o
02
= 0 since the corresponding transitions are not possible (see Figure
3.8).
For example, running this network in the slow change mode where only one mapping
is changed each time, corresponds to the initial state probability vector p = (0 0 1 0)

.
For a general network Γ(j
i
, j
n
, j
c
), A and p have the following form:
A =
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
o
00
o
01
0 ⋅ ⋅ ⋅ 0 0 0 0
o
10
o
11
o
12
⋅ ⋅ ⋅ 0 0 0 0
o
20
o
21
o
22
⋅ ⋅ ⋅ 0 0 0 0
0 o
31
o
32
⋅ ⋅ ⋅ 0 0 0 0
0 0 o
42
⋅ ⋅ ⋅ 0 0 0 0
0 0 0 ⋅ ⋅ ⋅ 0 0 0 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 ⋅ ⋅ ⋅ 0 0 0 0
0 0 0 ⋅ ⋅ ⋅ o
j
−4
j
−3
0 0 0
0 0 0 ⋅ ⋅ ⋅ o
j
−3
j
−3
o
j
−3
j
−2
0 0
0 0 0 ⋅ ⋅ ⋅ o
j
−2
j
−3
o
j
−2
j
−2
o
j
−2
j
−1
0
0 0 0 ⋅ ⋅ ⋅ o
j
−1
j
−3
o
j
−1
j
−2
o
j
−1
j
−1
0
0 0 0 ⋅ ⋅ ⋅ 0 o
j
j
−2
o
j
j
−1
1
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
p = (j
0
j
1
⋅ ⋅ ⋅ j
j
)

,
with o
(a−2) a
= o
(a−3) a
= ⋅ ⋅ ⋅ = o
0 a
= 0 and o
a+3 a
= ⋅ ⋅ ⋅ = o
j
a
= 0 since the
corresponding transitions are impossible (for appropriate values of n).
3.2.1 Statistical properties
The state transition matrix A and the initial state probability vector p enable to com
pute the statistics of the Markov chain in a very straightforward way. For detailed
analytical derivations see [13] and [17], for example.
28
For example, the elements of the nth power of A, noted A
a
, yield the transition
probabilities to a given state in n steps, i.e. o
a
na
is the transition probability from
state o
a
to state o
n
in n steps.
To motivate the above statement, consider the previous example of the network with
three inputs. The probability 1:
(2)
(o
2
∣o
1
) of going from state o
1
to state o
2
in two
steps is computed as follows:
1:
(2)
(o
2
∣o
1
) =1:(o
2
∣o
1
) 1:(o
1
∣o
1
) + 1:(o
2
∣o
2
) 1:(o
2
∣o
1
) + 1:(o
2
∣o
3
) 1:(o
3
∣o
1
)
(3.3)
which in terms of the transition matrix elements is written:
1:
(2)
(o
2
∣o
1
) = o
21
o
11
+ o
22
o
21
+ o
23
o
31
. (3.4)
The last expression is the product of the second row of A with the ﬁrst column of A,
which is the element (1, 2) of the product AA ≡ A
2
of A with itself.
An important observation is that the above computation relied explicitly in the deﬁn
ing properties of the Markov chain: step o(t + 1) is completely determined from step
o(t) and the state transition probability matrix A. This allowed to factor 1:
(2)
(o
2
∣o
1
)
in terms of the accessible intermediate states in Eq. (3.3) and associate the resulting
probabilities to the elements of A in Eq. (3.4). This will be useful when interpreting
the numerical evidence for the Markov chain representation.
The powers of matrix A lead to a straightforward computation of the learning times
distribution j(j). Since the last row of A
a
gives the transition probabilities to the
absorbing state o
j
in n steps when starting from each of the j
i
+1 states, the product
with p gives the probability of reaching the absorbing state in n steps when starting
from the initial state distribution p:
j (j ≤ n) = (0 1)

A
a
p, n ∈ ℕ
0
(3.5)
The transient states matrix Tcontains the transition probabilities between nonabsorbing
states, i.e. all states excluding the absorbing state o
j
.
A =
(
T 0
F 1
)
where F is submatrix row of transition probabilities to the ﬁnal state o
j
.
The powers of matrix T are obtained from A
a
and its elements t
(a)
i )
correspond to the
transition probability in n steps from the nonabsorbing states o
)
to the nonabsorbing
state o
i
.
A
a
=
(
T
a
0
∗ 1
)
where ∗ represents the last row of A
a
except the element 1.
The learning times distribution j(j) can be expressed in terms of the submatrix T
a
,
since the element , of the last row of A
a
is equal to one minus the sum of the ,th
29
column of T
a
, i.e. o
a
j
)
= 1 −
∑
j
−1
i=0
o
a
i )
= 1 −
∑
i
n
a
i )
.
Rewriting the ﬁrst part of Eq. 3.5 in terms of T
a
yields:
j (j ≤ n) = 1 −1

T
a
ˆ p, n ∈ ℕ
0
(3.6)
where ˆ p = (j
0
j
1
⋅ ⋅ ⋅ j
i−1
)

are the components of p except the probability of starting
in the ﬁnal state o
j
and 1 is the ones column vector of size j
i
−1.
An absorbing Markov chain has a number of interesting properties [13]:
∙ The matrix I −T has an inverse N = (I −T)
−1
, which is called the Fundamental
Matrix.
∙ N = I +T+T
2
+⋅ ⋅ ⋅
∙ The element n
i )
of matrix N is the expected number of times the chain is in state
o
i
, when starting in state o
)
, before reaching the absorbing state o
j
.
The last property yields the expectation of the learning time ⟨j⟩, since adding the en
tries of column , of N gives the expect number of times in all transient state before
reaching the absorbing state o
j
, when starting in state o
)
.
Therefore,
⟨j⟩ = 1

N ˆ p (3.7)
Higher order moments of the random variable j can be obtained from the expression
of the factorial moments [17]:
1[j (j −1) ⋅ ⋅ ⋅ (j −n + 1)] = n! 1

T
a−1
N
a
ˆ p
where 1[⋅] is the statistical expectation.
The expectation of the interference events ⟨i⟩ is easily derived from N, by noting
that the interference probability in a given state o
)
is given by the element o
)−1 )
of A,
i.e. 1:(i∣o
)
) = o
)−1 )
.
Therefore:
⟨i⟩ = v

N ˆ p (3.8)
where v = (0 o
0 1
⋅ ⋅ ⋅ o
0 j
−2
−1
)

is the column vector with the elements of A corre
sponding to interference events.
3.2.2 Markov chain representation: numerical evidence
To test the validity of the Markov chain representation the following experiment was
performed:
∙ A test set of networks was selected with input layer size j
i
ranging from one to
twelve input nodes.
∙ For each input layer size j
i
:
30
– The output layer size was j
i
+ 1.
– Three middle layer sizes corresponding to the three regimes (subcritical,
critical, supercritical) were used: j
sub
n
= 2 j
critical
n
and j
super
n
= j
critical
n
,2.
∙ Each network was simulated for a large number of times and the following statis
tics recorded:
– state transition counts
– initial state counts
– j (j) learning time distribution
– ⟨j⟩ average learning time
– ⟨i⟩ average interference events
The state transition and initial state counts are possible to extract since the
network evolves in discrete steps and for each discrete step the learning state can
be computed from the active conﬁguration.
∙ The measured state transition and initial state counts are normalised according
to Eqs. (3.1) and (3.2) yielding
2
the maximum likelihood estimators [6] for A
MLE
and p
MLE
, respectively.
∙ For each measured A
MLE
and p
MLE
the quantities j (j)
pred
, ⟨j⟩
pred
and ⟨i⟩
pred
are computed from Eqs. (3.6), (3.7) and (3.8) respectively.
∙ The measured values are compared to the computed values using the Normalised
Root Mean Square Error (NRMSE) and the KolmogorovSmirnov 1 statistic:
– Normalised root mean square error RMSE[A
pred
] =
√
(
∑
a
pred
−⟨a
⟩
)
2
a
max
−a
min
– KolmogorovSmirnov statistic 1 = arg max
a
∣j (A ≤ r
i
)
pred
−j (A ≤ r
i
)∣
In order to obtain enough state transition counts for each state to enable a good esti
mation A
MLE
, the mappings were presented to the network with a uniform initial state
probability (except for the state o
j
). Under this mapping change mode, the learning
time distribution j (j) has a diﬀerent shape than under the slow mapping change mode,
for example. The Figure 3.9 shows an example of measured learning time distributions
j (j) for two sample networks and the predicted distributions j (j)
pred
that are obtained
from Eq. (3.6).
The NRMSE for ⟨j⟩
pred
and ⟨i⟩
pred
was zero across the tested networks. The actual
value was below the numerical accuracy of the experiment as estimated by the inverse
of the number of samples in each simulation 1,·. Such NRMSE value is perhaps an
indication that the underlying method is not applicable to validate the Markov Chain
representation.
The KolmogorovSmirnov 1 statistic for j(j)
pred
computed from Eq. (3.6) is shown in
Figure 3.10.
2
The derivation of the maximum likelihood estimator for A follows the usual scheme of minimising
the data log likelihood and applying Lagrange multipliers for each element of A.
31
10
0
10
1
10
2
10
3
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
Γ(4,10,5) [runs:4e+5 uniform]
p
(
ρ
)
ρ
model
markov
(a) Supercritical network Γ(4, 10, 5).
10
0
10
1
10
2
10
3
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
Γ(12,312,13) [runs:1e+6 uniform]
p
(
ρ
)
ρ
model
markov
(b) Subcritical network Γ(12, 312, 13).
Figure 3.9 – The learning time distribution j(j) for a uniform initial state probability
(except for the state o
). The black line represents the predicted distribution j (j)
pred
that
is obtained from Eq. (3.6). The green line is the measured j (j).
0 2 4 6 8 10 12
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
Kolmogorov−Smirnov Statistic
D
Input Layer Size
super
critical
sub
Figure 3.10 – The KolmogorovSmirnov statistic 1 for j(j)
MLE
for the test network set,
with networks ranging from two input nodes to 12 input nodes. The statistic 1 is computed
from Eq. (3.6) and decreases with increasing input layer size for the networks considered.
For the networks considered, the KolmogorovSmirnov 1 statistic was usually higher
in the supercritical and critical regimes. With increasing input layer size, the statistic
1 decreases and converges for all three regimes. The abrupt reduction for the network
with two input nodes lead to additional measurements with larger output layers but
the results were very similar.
The value diﬀerences in the statistic 1 indicate that the Markov chain representa
tion is more accurate for larger networks. Smaller networks may be better represented
by other types of Markov chain or may even not allow a Markov chain representation
by not respecting the Markov chain properties, for example.
A direct estimation of the Markov chain order might clarify the above question. Due to
timing constraints this was not pursued. Several estimators exist for the Markov chain
order, such as the BIC Markov order estimator [11] or the PeresShield order estimator
[19].
32
3.2.3 Analytical solution for Γ(2,
n
,
c
)
An analytical solution for A has been obtained for network with two input nodes
Γ(2, j
n
, j
c
), for the case where the input nodes cannot share output nodes
3
.
For Γ(2, j
n
, j
c
) the state transition matrix A and the initial state transition vector
A have the form:
A =
⎛
⎝
o
00
o
01
0
o
10
o
11
0
o
20
o
21
1
⎞
⎠
and p = (j
0
j
1
j
2
)

, (3.9)
Each state o
a
can be further separated in substates o
o
a
corresponding to the graphs
with d degrees of freedom in the middle layer. The basic graphs o
o
a
for Γ(2, j
n
, j
c
) are
illustrated in Figure 3.11. For simplicity, these substates are simply referred as the
graphs of state state o
a
.
Figure 3.11 – The basic graphs for the two input networks Γ(2, j
, j
). The upper index
speciﬁes the number of degrees of freedom in the middle layer, i.e. o
2
1
is the graph for one
learnt input node and one shared middle layer node.
The main steps in this computation are the following:
1. Compute the number of active conﬁgurations C for each state o
a
in terms of the
graphs o
o
a
.
2. Equal graph accessibility approximation: assume that in a given state o
a
each
of graphs o
o
a
is equally accessible. This is strictly the case for o(0) where the
distribution of graphs is only dependent on the relative proportion of active con
ﬁgurations C implementing each graph. However, it is not necessarily the case
for transitions from another state o
˜ a
or from the same state o
a
where the graph
to graph transitions may favour some particular graphs.
3. For each graph o
o
a
compute the probability of transition to o
a−1
, o
a
, o
a+1
and
o
a+2
corresponding to unlearning, no change, learning one and two mappings,
respectively.
3
The case where the input nodes can share output nodes may be easily derived by simpliﬁcation of
the present results.
33
When starting the network from random weights, the active conﬁguration distribution
for each state o
a
does not correspond to the relative frequency count of its graphs, as
shown in Table 3.2.
Graphs
o
1
0
o
1
1
o
1
2
o
2
0
o
2
1
o
2
2
Conﬁguration count
6 6 0
54 36 6
Conﬁguration count distribution
0.0556 0.0556 0
0.5000 0.3333 0.0556
Random graph distribution
0.1666 0.1666 0
0.3749 0.2502 0.0417
Table 3.2 – The conﬁguration count distribution and the initial graph distribution when
starting from random weights do not match. The values above are for the network Γ(2, 3, 4).
The conﬁguration count distribution is obtained by counting the relative frequency of the
conﬁgurations implementing a given graph. For example, Γ(2, 3, 4) has six possible conﬁg
urations for graph o
1
0
out of 108 distinct conﬁgurations, which gives a relative frequency of
0.0556 for graph o
1
0
.
The correct distribution is obtained by ﬁrst counting the conﬁgurations generated from
nonshared middle layer nodes and one of the shared middle layer nodes and then
multiplying by j
c
for each shared additional shared middle layer node. Such strategy is
justiﬁed by considering all the combinations of triplets (i, :, o) and enforcing the WTA
rule in the second and third value of the triplet. The total number of combinations is
(j
n
j
c
)
j
and the WTA rule assigns them to the corresponding graphs o
o
a
. The resulting
graph distribution is the correct one, as shown in Table 3.3. This approach has also
been veriﬁed successfully for the network Γ(3, j
n
, j
c
).
Random graph distribution
0.1666 0.1666 0
0.3749 0.2502 0.0417
Predicted graph distribution
0.1667 0.1667 0
0.3750 0.2500 0.0417
Table 3.3 – The initial graph distribution when starting from random weights matches the
predicted graph distribution, for the network Γ(2, 3, 4). The predicted graph distribution is
detailed in the text.
For the network Γ(2, j
n
, j
c
) one obtains the following number of conﬁgurations per
graph:
o
1
0
= j
n
j
c
(j
c
−2)
o
1
1
= j
i
j
n
j
c
o
2
0
= j
n
(j
n
−1) (j
c
−1)
2
o
2
1
= j
i
j
n
(j
n
−1) (j
c
−1)
o
2
2
= j
n
(j
n
−1)
where
o
o
a
is the order of the graph o
o
a
, i.e. the number of distinct conﬁgurations that
implement the graph.
34
This allows to compute p
rand
when starting from a random conﬁguration:
p
rand
=
1
∣o∣
(
o
1
0
+
o
2
0
o
1
1
+
o
2
1
o
2
2
)
where ∣o∣ ≡
o
1
0
+
o
2
0
+
o
1
1
+
o
2
1
+
o
2
2
.
One ﬁnal element before computing the transition probabilities is to determine the
probability j
0
of reselecting the same node after depression and the probability of se
lecting a diﬀerent node 1 −j
0
. The reselection probability is given by j
0
= 1,(2 j
a
−1)
and this value can only be related to the particular weight distribution of this model,
as shown in Figure 2.5b. Numerical evidence for such value of j
0
is shown in Figure
3.12.
0 10 20 30 40
0
0.1
0.2
0.3
0.4
0.5
Average Re−selection Probability
Γ(2,*,10) [depressions:1e+4]
〈
P
r
(
r
e
−
s
e
l
e
c
t
i
o
n
)
〉
Middle Layer Size
re−select
1/(2η
n
−1)
1/η
n
Figure 3.12 – The average reselection probability in the middle layer nodes, for the
networks Γ(2, j
, 10). The measured probability (in blue) follows the curve 1,(2 j
−1) (in
green). The curve 1,j
is shown for comparison (in red). Data collected for 1c4 depressions
in each network.
The element o
01
of the transition matrix A for Γ(2, j
n
, j
c
) can be obtained as follows:
o
01
=1:(o
0
∣o
1
) = 1:(o
1
0
∣o
1
1
) 1:(o
1
1
∣o
1
) + 1:(o
2
0
∣o
1
1
) 1:(o
1
1
∣o
1
)
Using the equal graph accessibility approximation,
o
01
≃
(
1:(o
1
0
∣o
1
1
) + 1:(o
2
0
∣o
1
1
)
)
o
1
1
o
1
1
+
o
2
1
which with the graph transition probabilities gives:
o
01
≃
(
:
0
(1 −o
0
)
j
c
−2
j
c
−1
+ (1 −:
0
)(1 −o
0
)
j
c
−1
j
c
)
o
1
1
o
1
1
+
o
2
1
where :
0
and o
0
represent the reselection probabilities of middle and output nodes,
respectively.
The main elements of the last computation are detailed below, where it is assumed
the network is learning to connect input two to output two while learning the iden
tity mapping ` = {1, 2} (the other possible mapping set requiring an inversion of the
roles).
35
∙
∣S
1
1
∣
∣S
1
1
∣+∣S
2
1
∣
is the equal graph accessibility approximation for graphs o
1
1
and o
2
1
in
state o
1
.
∙ :
0
(1−o
0
)
j
−2
j
−1
is the transition probability from o
1
1
to o
1
0
. This transition occurs
whenever the same middle layer node is reselected and the output node is not
reselected. There are j
c
−2 out of j
c
−1 such cases that enable the transition.
∙ (1 −:
0
)(1 −o
0
)
j
−1
j
is the transition probability from o
1
1
to o
2
0
, occurring when
ever the same middle layer node is not reselected, the output node is not re
selected and there are j
c
−1 out of j
c
cases that enable the transition.
Figure 3.13 – The graph transitions from o
1
1
to o
1
0
and o
2
0
. The target mapping is ` =
{1, 2}.
3.2.4 Alternate formulation: graph transitions
There is an alternative to relying on the equal graph accessibility approximation. By
considering the transitions between graphs o
o
a
directly rather than between states o
a
,
the need to compute the graph occupation within states is avoided altogether.
The resulting transition matrix has more elements, one column for each possible graph,
but the predictions are potentially more accurate.
The elements of this new state transition matrix A
j
and the new initial probabilities
vector p
j
, are as follows:
A
j
=
⎛
⎜
⎜
⎜
⎜
⎝
o
10 10
o
10 20
o
10 11
0 0
o
20 10
o
20 20
o
20 11
0 0
o
11 10
o
11 20
o
11 11
o
11 21
0
o
21 10
o
21 20
o
21 11
o
21 21
0
o
22 10
0 o
22 11
o
22 21
1
⎞
⎟
⎟
⎟
⎟
⎠
and p
j
= (j
10
j
20
j
11
j
21
j
22
)

,
where o
22 20
= o
10 21
= o
20 21
= 0 since no transitions are possible between those graphs.
The element o
¸aoa
represents 1:(o
¸
a
∣o
o
a
), the transition probability from graph o
o
a
to graph o
¸
a
.
36
Figure 3.14 – The graph transitions for a network with two input nodes. The arrows
indicate the sense of the transitions. No transition is possible from graph o
2
2
to any of the
other graph.
The analytical results previously obtained are directly applicable to compute the el
ements of the transition matrix A
j
. For example, the element o
01
of A yields the
elements o
10 11
and o
20 11
of A
j
, as follows:
o
10 11
=1:(o
1
0
∣o
1
1
) = :
0
(1 −o
0
)
j
c
−2
j
c
−1
o
20 11
=1:(o
2
0
∣o
1
1
) = (1 −:
0
)(1 −o
0
)
j
c
−1
j
c
These two elements of A
j
correspond to the two graph transitions in Figure 3.13.
Computing j (j) and ⟨j⟩ is according to Eqs. (3.6) and (3.7) respectively. However, Eq.
(3.8) is no longer applicable for computing ⟨i⟩, as the elements of A
j
corresponding to
interference events are no longer on the upper diagonal of matrix A
j
.
Nevertheless, a similar computation is still possible:
⟨i⟩ =
∑
j:column
of N
∑
i:interference
graph of j
o
j
i )
n
i )
ˆ j
)
(3.10)
where o
j
i )
is an element of A
j
corresponding to an interference transition from graph
, to graph i, ˆ p
j
is the initial state transition vector excluding the absorbing graph and
ˆ j
)
is the element , of ˆ p
j
.
3.2.5 Analytical solution: numerical evidence
To test the validity of the analytical solution, a similar methodology was used as in
Subsection 3.2.2 for testing the Markov chain representation. In the present case, the
network test set was limited to networks with two input nodes, 10 output nodes and a
range of middle layer nodes.
The analytical expressions for A and A
j
were used to obtain j(j)
j·co
, ⟨j⟩
j·co
and
⟨i⟩
j·co
from Eqs. (3.6), (3.7) and (3.8) or (3.10), respectively, and the predicted values
were then compared to the ones obtained from the simulations.
37
The slow mapping change mode was used this time, instead of the uniform initial
state probability, as the number of states is quite small and the slow change mode is
able to visit them a suﬃcient number of times. The resulting learning time distribu
tions j(j) are more representative of the typical network dynamics.
10
0
10
1
10
2
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
Gamma(2,10,10) [runs:1e+6 slow]
p
(
ρ
)
ρ
model
pred A
pred A
g
(a) Supercritical network Γ(2, 10, 10).
10
0
10
1
10
2
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
Gamma(2,100,10) [runs:1e+6 slow]
p
(
ρ
)
ρ
model
pred A
pred A
g
(b) Subcritical network Γ(2, 100, 10).
Figure 3.15 – The learning time distribution j(j) in the slow change mode. The green
and red lines are the prediction from Eq. (3.6) using A and A
respectively.
The predicted learning time distributions j(j)
j·co
are quite similar to the ones mea
sured. This can be appreciated from Figure 3.15, where the predictions are compared
in the supercritical and the subcritical regimes. As with the predictions from the
Markov chain validation testing, the supercritical regime distributions are less accu
rate than the subcritical distributions.
0 20 40 60 80 100 120
0
0.01
0.02
0.03
0.04
0.05
0.06
Kolmogorov−Smirnov Statistic
Gamma(2,*,10) [runs:1e+6 slow]
D
Middle Layer Size
pred A
pred A
g
Figure 3.16 – The KolmogorovSmirnov statistic for the learning time distribution j(j)
in the slow change mode. The green and red lines are the statistic value obtained from Eq.
(3.6) using A and A
respectively.
The above is conﬁrmed by the KolmogorovSmirnov 1 statistic, which is higher for the
supercritical and critical regimes, as show in Figure 3.16.
The statistic 1 decreases with increasing input layer size and is comparable to the
maximum likelihood estimations for the transition matrix, obtained in Subsection
38
3.2.2 for the critical and subcritical regimes (compare Figure 3.16 with the D value for
network size two from Figure 3.10).
Interestingly, the predictions obtained from A
j
don’t have better performance than
the ones from A, except in the subcritical regime where the 1 statistic is lower for the
j(j)
j·co
obtained from A
j
. This may be particular to the two input networks, which
are somewhat singled out from the others in Figure 3.10.
In terms of ⟨j⟩
pred
and ⟨i⟩
pred
, the predictions obtained from A are also more ac
curate than the ones obtained from A
j
, as shown in Table 3.4 and Figure 3.17.
0 20 40 60 80 100 120
10
15
20
25
30
Average Learning Time 〈ρ〉
Gamma(2,*,10) [runs:1e+6 slow]
〈
ρ
〉
Middle Layer Size
model
pred A
pred A
g
0 20 40 60 80 100 120
0
0.5
1
1.5
Average Interference Event 〈ν〉
Gamma(2,*,120) [runs:1e+6 slow]
〈
ν
〉
Middle Layer Size
model
pred A
pred A
g
Figure 3.17 – Comparing ⟨j⟩
pred
and ⟨i⟩
pred
to the measured values, in the slow change
mode. The green and red lines are the prediction from Eq. (3.7) and (3.10) for A and A
respectively.
The improvement in the KolmogorovSmirnov 1 statistic for A
j
in the subcritical
regime did not reﬂect in the NRMSE, where the predictions from A are consistently
more accurate.
NMRSE ⟨j⟩
pred
supercritical critical subcritical
A 0.0839 (0.0660) 0.0577 (0) 0.0149 (0.0203)
A
j
0.1042 (0.0758) 0.0817 (0) 0.0264 (0.0332)
NMRSE ⟨i⟩
pred
supercritical critical subcritical
A 0.1323 (0.1066) 0.1339 (0) 0.0452 (0.0616)
A
j
0.1704 (0.1269) 0.2000 (0) 0.0875 (0.1089)
Table 3.4 – The normalised root mean square error (NMRSE) for the predicted ⟨j⟩
pred
and ⟨i⟩
pred
in the supercritical, critical and subcritical regimes. The values in brackets
are the standard deviation on the error NMRSE.
3.2.6 Summary
The key elements of this section are the following:
39
∙ Markov chain representation: The ChialvoBak network in the twolayer
topology is fairly accurately represented by a ﬁrst order Markov chain, where the
chain states correspond to the number of learnt maps.
∙ Markov chain statistics: The state transition matrix A obtained either ana
lytically or by maximum likelihood estimation, easily allows to compute j (j)
pred
,
⟨j⟩
pred
and ⟨i⟩
pred
.
∙ Numerical evidence for Markov chain: The Markov chain representation is
accurate in all regimes (supercritical, critical and subcritical) for all but very
small networks.
∙ Analytical solution for A for Γ(2, j
n
, j
c
): An approximate analytical solution
was obtained for the network of two input nodes.
∙ Analytical solution for A
j
for Γ(2, j
n
, j
c
): An analytical solution was ob
tained for the network of two input nodes when considering the transitions be
tween graphs o
o
a
rather than between states. As such, this solution is in principle
exact.
∙ Numerical evidence for analytical solution: The performance of the analyt
ical solutions is comparable to a maximum likelihood estimation of the transition
matrix A in the critical and subcritical regimes. The predictions from the ap
proximate solution had better performance than the nonapproximate solution.
3.2.7 Appendix
Analytical Solution with A for Γ(2, j
n
, j
c
)
The solution for the full transition matrix A for Γ(2, j
n
, j
c
) is as follows:
o
00
≃
o
1
0
o
1
0
+
o
2
0
(
1:(o
1
0
∣o
1
0
) + 1:(o
2
0
∣o
1
0
)
)
+
o
2
0
o
1
0
+
o
2
0
(
1:(o
1
0
∣o
2
0
) + 1:(o
2
0
∣o
2
0
)
)
≃
o
1
0
o
1
0
+
o
2
0
(
:
0
(
o
0
+ (1 −o
0
)
j
c
−3
j
c
−1
)
+ (1 −:
0
)
j
c
−1
j
c
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
))
+
o
2
0
o
1
0
+
o
2
0
:
0
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
+
o
2
0
o
1
0
+
o
2
0
(
(1 −:
0
)
j
n
−2
j
n
−1
j
c
−1
j
c
+ (1 −:
0
)
1
j
n
−1
j
c
−2
j
c
−1
)
o
10
≃
o
1
0
o
1
0
+
o
2
0
(
1:(o
1
1
∣o
1
0
) + 1:(o
2
1
∣o
1
0
)
)
+
o
2
0
o
1
0
+
o
2
0
(
1:(o
1
0
∣o
2
0
) + 1:(o
2
0
∣o
2
0
)
)
≃
o
1
0
o
1
0
+
o
2
0
:
0
(1 −o
0
)
2
j
c
−1
+
o
1
0
o
1
0
+
o
2
0
(
(1 −:
0
)
(
1
j
c
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
+
j
0
−1
j
0
(1 −o
0
)
1
j
c
−1
))
+
o
2
0
o
1
0
+
o
2
0
(
:
0
(1 −o
0
)
1
j
c
−1
+ (1 −:
0
)
j
n
−2
j
n
−1
1
j
c
+ (1 −:
0
)
1
j
n
−1
1
j
c
−1
)
40
o
20
≃
o
1
0
o
1
0
+
o
2
0
1:(o
2
2
∣o
1
0
) ≃
o
1
0
o
1
0
+
o
2
0
(
(1 −:
0
)
1
j
c
(1 −o
0
)
1
j
c
−1
)
o
01
≃
o
1
1
o
1
1
+
o
2
1
(
1:(o
1
0
∣o
1
1
) + 1:(o
2
0
∣o
1
1
)
)
≃
o
1
1
o
1
1
+
o
2
1
(
:
0
(1 −o
0
)
j
c
−2
j
c
−1
+ (1 −:
0
)(1 −o
0
)
j
c
−1
j
c
)
o
11
≃
o
1
1
o
1
1
+
o
2
1
(
1:(o
1
1
∣o
1
1
) + 1:(o
2
1
∣o
1
1
)
)
+
o
2
1
o
1
1
+
o
2
1
(
1:(o
2
1
∣o
2
1
) + 1:(o
1
1
∣o
2
1
)
)
≃
o
1
1
o
1
1
+
o
2
1
(
:
0
(
o
0
+ (1 −o
0
)
1
j
c
−1
)
+ (1 −:
0
)
(
1
j
c
(1 −o
0
) +
j
c
−1
j
c
o
0
))
+
o
2
1
o
1
1
+
o
2
1
:
0
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
+
o
2
1
o
1
1
+
o
2
1
+
(
(1 −:
0
)
j
n
−2
j
n
−1
j
c
−1
j
c
+ (1 −:
0
)
1
j
n
−1
)
o
22
≃
o
1
1
o
1
1
+
o
2
1
1:(o
2
2
∣o
1
1
) +
o
2
1
o
1
1
+
o
2
1
1:(o
2
2
∣o
2
1
)
≃
o
1
1
o
1
1
+
o
2
1
(
(1 −:
0
)
1
j
c
o
0
)
+
o
2
1
o
1
1
+
o
2
1
(
:
0
(1 −o
0
)
1
j
c
−1
+ (1 −:
0
)
j
n
−2
j
n
−1
1
j
c
)
o
02
=0 o
12
= 0 o
22
= 1
where :
0
and o
0
represent the reselection probabilities of middle and output nodes,
respectively.
Analytical Solution with A
j
for Γ(2, j
n
, j
c
)
The solution for the full transition matrix A
j
for Γ(2, j
n
, j
c
) is as follows: where :
0
and o
0
represent the reselection probabilities of middle and output nodes, respectively.
o
10 10
=1:(o
1
0
∣o
1
0
) = :
0
(
o
0
+ (1 −o
0
)
j
c
−3
j
c
−1
)
o
20 10
=1:(o
2
0
∣o
1
0
) = (1 −:
0
)
j
c
−1
j
c
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
o
11 10
=1:(o
1
1
∣o
1
0
) = :
0
(1 −o
0
)
2
j
c
−1
o
21 10
=1:(o
2
1
∣o
1
0
) = (1 −:
0
)
(
1
j
c
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
+
j
0
−1
j
0
(1 −o
0
)
1
j
c
−1
)
o
22 10
=1:(o
2
2
∣o
1
0
) = (1 −:
0
)
1
j
c
(1 −o
0
)
1
j
c
−1
41
o
10 20
=1:(o
1
0
∣o
2
0
) = (1 −:
0
)
1
j
n
−1
j
c
−2
j
c
−1
o
20 20
=1:(o
2
0
∣o
2
0
) = :
0
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
+ (1 −:
0
)
j
c
−2
j
c
−1
j
c
−1
j
c
o
11 20
=1:(o
1
1
∣o
2
0
) = (1 −:
0
)
1
j
n
−1
1
j
c
−1
o
21 20
=1:(o
2
1
∣o
2
0
) = :
0
(1 −o
0
)
1
j
c
−1
+ (1 −:
0
)
j
n
−2
j
n
−1
1
j
c
o
22 20
=1:(o
2
2
∣o
2
0
) = 0
o
10 11
=1:(o
1
0
∣o
1
1
) = :
0
(1 −o
0
)
j
c
−2
j
c
−1
o
20 11
=1:(o
2
0
∣o
1
1
) = (1 −:
0
)(1 −o
0
)
j
c
−1
j
c
o
11 11
=1:(o
1
1
∣o
1
1
) = :
0
(
o
0
+ (1 −o
0
)
1
j
c
−1
)
o
21 11
=1:(o
2
1
∣o
1
1
) = (1 −:
0
)
(
1
j
c
(1 −o
0
) +
j
c
−1
j
c
o
0
)
o
22 11
=1:(o
2
2
∣o
1
1
) = (1 −:
0
)
1
j
c
o
0
o
10 21
=1:(o
1
0
∣o
2
1
) = 0
o
20 21
=1:(o
2
0
∣o
2
1
) = 0
o
11 21
=1:(o
1
1
∣o
2
1
) = (1 −:
0
)
1
j
n
−1
o
21 21
=1:(o
2
1
∣o
2
1
) = :
0
(
o
0
+ (1 −o
0
)
j
c
−2
j
c
−1
)
+ (1 −:
0
)
j
n
−2
j
n
−1
j
c
−1
j
c
o
22 21
=1:(o
2
2
∣o
2
1
) = :
0
(1 −o
0
)
1
j
c
−1
+ (1 −:
0
)
j
n
−2
j
n
−1
1
j
c
o
10 22
=0 o
20 22
= 0 o
11 22
= 0 o
21 22
= 0 o
22 22
= 1
3.3 Learning Convergence
In all but very small networks, the two layer topology of the ChialvoBak model seems
well represented by a ﬁrstorder absorbing Markov chain. This is supported by the
numerical testing results of the Markov chain representation.
An important property of absorbing Markov Chains is the guaranteed convergence
4
to the absorbing states. Therefore, as long as the absorbing Markov chain represen
tation holds, the corresponding network can be expected to converge to the complete
learning state.
More speciﬁcally, for a given learning rule, the network is expected to converge to
the complete learning state if under that learning rule:
4
This property is based on a probability conservation argument. For details see [13], for example.
42
∙ The Markov property is preserved, i.e.
1:(o(t + 1) ∣ o(t), o(t −1), ⋅ ⋅ ⋅ , o(0)) = 1:(o(t + 1) ∣ o(t))
∙ From every possible state o
a
there is a path to the absorbing state o
j
, which
does not need to be a direct path.
A necessary condition seems to be a requirement to have separate random depressions
in the middle and output layers. Indeed, simulations showed that for the network
Γ(2, 2, 2) complete learning is not guaranteed if the middle and output layers are de
pressed by the same random amount. This condition seems to be needed to maintain
the Markov property, but further investigation is necessary to verify this.
For any state o
a
to be able to reach the absorbing state o
j
it is suﬃcient that at
least one path to the absorbing state exists. Such path can be constructed if for every
state o
a
, a transition is possible to a higher learning state o
a+1
. This translates to a
lower bound on the number of middle layer nodes. For the system to be in the learning
state o
a
, a total of n middle layer nodes are used to support this state. Moving to the
state o
a+1
requires one additional free middle layer node, and so on.
For the simple learning rule, the number of middle layer nodes should therefore be
as large as the number of input nodes, i.e. j
n
≥ j
i
.
For the advanced learning rule, the amount of middle layer nodes should to be as
large as the number of diﬀerent input conﬁgurations, i.e. j
n
≥ ∣1∣, where ∣1∣ represents
the order of the input conﬁgurations 1.
Proving the above assertions would be very interesting, as it would establish the learn
ing capability in the advanced learning rule, which is the ability to learn arbitrary
binary inputs to output maps, and in particular, any boolean function.
3.3.1 Summary
The key elements of this section are the following:
∙ Guaranteed convergence of the Markov chain: An absorbing Markov chain
is guaranteed to converge to the absorbing state. Should the Markov chain repre
sentation hold for the simple and advanced learning modes, the ChialvoBak can
be expected to converge to the complete learning state.
∙ Separate random depressions: A requirement for convergence is to have sep
arate random depressions in the middle and output layers. This seems to be
related to maintaining the Markov property.
∙ Lower bound on the number of middle layer nodes : Another requirement
for learning convergence is to have suﬃcient middle layer nodes. For the simple
learning rule j
n
≥ j
i
and for the advanced learning rule j
n
≥ ∣1∣, where ∣1∣
represents the order of the input conﬁgurations 1.
43
3.4 PowerLaw Behaviour and Neural Avalanches
The power law tail
5
in the learning time distribution j(j) of the critical regime is also
dependent on the mapping change mode, i.e. the way new mapping sets are presented
to the network.
The power law tails of the distributions in Figure 2.8a were generated under the slow
change mode, where one single mapping is changed each time. It turns out that the slow
change mode seems to be the only mapping presentation mode that results in power
law behaviour. If the network is provided with random mapping sets the powerlaw
disappears, as illustrated in Figure 3.18a.
10
1
10
2
10
3
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
Γ(8,72,9) [runs:1e+6]
p
(
ρ
)
ρ
slow
rand
(a) Learning under slow change mode and ran
dom change mode. The blue line represents
learning with one mapping change each time,
whereas the green line represents learning ran
dom mapping sets.
10
1
10
2
10
3
10
−4
10
−2
10
0
Learning Time Distribution p(ρ)
Γ(8,72,9) [synthetic]
P
(
ρ
)
ρ
S
1
S
4
S
6
S
7
(b) Learning under j
≥ n ≥ 1 mapping
changes, i.e. the slow change mode corre
sponds to n = 1. Learning under n mapping
changes corresponds to starting the system in
state o
−
. The plot was generated from syn
thetic data obtained from the A
MLE
.
Figure 3.18 – The powerlaw tail of the distribution j(j) disappears when presenting
mappings to the network in other ways than the slow change mode.
In the Markov chain representation, the slow change mode is equivalent to starting the
network in state o(0) = o
j
−1
. As such, learning under j
i
≥ n ≥ 1 mapping changes
amounts to starting the network in the state o(0) = o
j
−a
. Again, only the slow change
mode generated a power law tail, as illustrated in Figure 3.18b.
An interesting characterisation of the critical regime is obtained by analysing the state
occupation matrix Q that is obtained from the transition counts after running a simula
tion. Normalising the transition counts globally gives the frequency of each transition,
and summing over the rows gives the state frequency. Below is an example for the a
network Γ(6, 42, 7) after 1c5 runs in the slow change mode.
5
As mentioned in SubSection 2.1.3 the approximately straight segments in the distributions of
Figure 2.8b do not necessarily imply that j(j) is a proper power law tail distribution. Due to timing
constraints no conclusive power law testing was completed for j(j), as such the terminology proposed
in [20] is adopted here.
44
Q =
⎛
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎜
⎝
0.0063 0.0011 0 0 0 0
0.0011 0.0345 0.0059 0 0 0
0.0000 0.0059 0.0950 0.0154 0 0
0 0.0000 0.0153 0.1704 0.0262 0
0 0 0.0001 0.0261 0.2255 0.0328
0 0 0 0.0001 0.0327 0.2683
0 0 0 0 0.0000 0.0373
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎠
Summing over the columns of Q gives the relative occupation in each state. For the
network Γ(6, 42, 7) the relative occupation of each state is shown in Table 3.5.
Analysing the columns of Q for larger networks yields the following picture:
∙ In the supercritical regime the network spends most of the time in the states
neighbouring o
j
¸2
.
∙ In the critical regime the network spends most of the time in the states o
j
¸2
, ⋅ ⋅ ⋅ o
j
−1
.
∙ In the subcritical regime the network spends most of the time in the last two
states o
j
−1
and o
j
−2
.
Relative State Occupation
o
0
o
1
o
2
o
3
o
4
o
5
supercritical 0.0648 0.1971 0.2800 0.2437 0.1449 0.0695
critical 0.0075 0.0416 0.1163 0.2119 0.2844 0.3384
subcritical 0.0004 0.0038 0.0218 0.0849 0.2431 0.6460
Table 3.5 – The relative state occupation of the network Γ(6, 42, 7) after 1c5 runs in the
slow change mode. In the supercritical regimes the network spends most time in the middle
states. In the critical regime the occupation is from the middle state up to the state before
absorption. In the subcritical regime the network spends most of the time in the last two
states before absorption.
3.4.1 Biological interpretation
The slow change mode may have an interesting biological interpretation. Assigning a
slow temporal scale to the rate of mapping changes, slow enough so that these would
be comparable to the perturbations in the synaptic weights resulting from biological
background noise, would allow to reinterpret the slow change mode as the network
recovering from noise.
Small punctual perturbations to the synaptic weights might aﬀect the result of the
WTA rule for a given node of the active conﬁguration. This is likely to result in the
activation of a diﬀerent output node, which would trigger the corrective depressions.
This eﬀect is comparable to reassigning an output node, specially if the perturbation
aﬀects a synapse from an input node to a middle layer node. The ability of the network
to recover perfectly from noise has been discussed in [8][1].
45
The neural avalanches, ﬁrst evidenced experimentally J. Beggs and D. Plenz [4][5] cor
respond to spontaneous neural activity displaying powerlaw distributions in the event
sizes. Identifying both avalanche types is not trivial as the biological neural networks
analysed by Beggs and Plenz were not know to be performing any particular activity
during the experiments, whereas the ChialvoBak avalanches are due to an ongoing
process of synaptic plasticity.
As such, a parallel may be established between the biological neural avalanches and
the avalanches in the ChialvoBak model the slow change mode, under a very slow
temporal scale of changes.
3.4.2 Summary
The key elements of this section are the following:
∙ Power law tail on slow change mode only: The power law tail in the learning
time distribution 1(j) only seems to occur in the slow change mode.
∙ Relative state occupation in the slow change mode: In the supercritical
regimes the network spends most time in the middle states, whereas in the critical
regime the occupation is from the middle state up to the state before absorption.
In the subcritical regime the network spends most of the time in the last two
states before absorption.
∙ Parallel with neural avalanches: By reinterpreting the slow change mode
under a very slow temporal scale of changes, as the network recovering from
noise.
46
Chapter 4
Conclusion
The present work focused on extending the analytical understanding of the Chialvo
Bak network in the twolayer topology.
The model seems to be accurately represented by a ﬁrst order Markov chain, where the
chain states correspond to the number of learnt maps. This representation easily allows
to compute important statistics of the network, such as j (j)
pred
, ⟨j⟩
pred
and ⟨i⟩
pred
.
Numerical testing found support for the Markov chain representation in all but very
small networks.
A direct estimation of the Markov chain order would further contribute to the un
derstanding of the Markov chain representation and its eventual limitations.
An analytical solution was developed for the transition matrix A in the case of a
network with two input nodes Γ(2, j
n
, j
c
). The performance of the analytical solu
tions is comparable to the maximum likelihood estimation obtained from simulations
of Γ(2, j
n
, j
c
) in the critical and subcritical regimes.
Extending analytical for networks with higher number of inputs would consolidate the
Markov chain representation and could lead to a better understanding of the critical
regime.
Should the Markov chain representation hold in general, for the simple and advanced
learning modes, then the model can be expected to converge to the complete learning
state. A convergence requirement is to have separate random depressions in the middle
and output layers, which seems to be related to maintaining the Markov property. A
lower bound on the number of middle layer nodes is required for maintaining the ab
sorbing property of the Markov chain.
A proper analytical proof of the learning convergence would be useful to establish
the learning capability of the model.
The power law tail in the learning time distribution 1(j) seems limited to the slow
change mode. The Markov chain representation provides a characterisation of the crit
ical regime, for which the network spends most of the time in the state where half the
mappings are learnt, up to the state before absorption. Very rarely all the mappings
are unlearnt in the critical regime.
47
A systematic power law testing of the learning time distribution j(j) in the critical
regime is necessary to establish the power law tail nature of this distribution.
A parallel with the biological neural avalanches is proposed by reinterpreting the slow
change mode under very slow temporal scale of changes, as the network recovering from
noise rather than learning new mappings.
The permanent tagging of successful weights by the selective punishment rule results
in reduced performance for large timescales. A global tag threshold, which places a
global limit on the number of times a tagged weight can be wrong before becoming
untagged, seems successful in eliminating the ageing eﬀect and is independent on the
level of network activity.
The local tag threshold was not developed in the present work, however, its local
character potentially results in network size independence, which is a clear advantage
over the global tag threshold.
48
Bibliography
[1] P. Bak and D.R. Chialvo. Adaptive learning by extremal dynamics and negative
feedback. Arxiv preprint condmat/0009211, 2000.
[2] Per Bak. How Nature Works: The science of SelfOrganized Criticality. Springer
Verlag, 1996.
[3] A. Barr, E.A. Feigenbaum, and P.R. Cohen. The handbook of artiﬁcial intelligence.
AddisonWesley Reading, MA, 1989.
[4] J.M. Beggs and D. Plenz. Neuronal avalanches in neocortical circuits. Journal of
Neuroscience, 23(35):11167–11177, 2003.
[5] J.M. Beggs and D. Plenz. Neuronal avalanches are diverse and precise activity
patterns that are stable for many hours in cortical slice cultures. Journal of neu
roscience, 24(22):5216–5229, 2004.
[6] P. Billingsley. Statistical inference for Markov processes. University of Chicago
Press, 1961.
[7] RJC Bosman, WA van Leeuwen, and B. Wemmenhove. Combining Hebbian and
reinforcement learning in a minibrain model. Neural Networks, 17(1):29–36, 2004.
[8] DR Chialvo and P. Bak. Learning from mistakes. Neuroscience, 90(4):1137–1148,
1999.
[9] A. Clauset, C.R. Shalizi, and M.E.J. Newman. Powerlaw distributions in empirical
data. arxiv, 706, 2007.
[10] F. Crepel, N. Hemart, D. Jaillard, and H. Daniel. Longterm depression in the
cerebellum. Handbook of Brain Theory and Neural Networks, 1998.
[11] I. Csiszar and P.C. Shields. The consistency of the BIC Markov order estimator.
Annals of Statistics, pages 1601–1619, 2000.
[12] P. Dayan, L.F. Abbott, and L. Abbott. Theoretical neuroscience: Computational
and mathematical modeling of neural systems. MIT Press, 2001.
[13] C.M. Grinstead and J.L. Snell. Introduction to probability. Amer Mathematical
Society, 1997.
[14] M. Ito. Longterm depression. Annual Review of Neuroscience, 12(1):85–102, 1989.
[15] M.H. Johnson. Developmental Cognitive Neuroscience: An Introduction. Blackwell
Publishing, 1997.
49
[16] K. Klemm, S. Bornholdt, and H.G. Schuster. Beyond Hebb: exclusiveOR and
biological learning. Physical Review Letters, 84(13):3013–3016, 2000.
[17] G. Latouche and V. Ramaswami. Introduction to matrix analytic methods in
stochastic modeling. Society for Industrial Mathematics, 1999.
[18] M. Minsky. Steps toward artiﬁcial intelligence. Proceedings of the IRE, 49(1):8–30,
1961.
[19] Y. Peres and P. Shields. Two new Markov order estimators. Arxiv preprint
math/0506080, 2005.
[20] J. Wakeling. Order–disorder transition in the Chialvo–Bak ”minibrain” controlled
by network geometry. Physica A: Statistical Mechanics and its Applications, 325(3
4):561–569, 2003.
[21] J.R. Wakeling. Adaptivity and ”Per learning”. Physica A: Statistical Mechanics
and its Applications, 340(4):766–773, 2004.
50