Adaptive resonance theory

Adaptive resonance theory (ART) is a theory developed by Stephen Grossberg and Gail Carpenter on
aspects of how the brain processes information. It describes a number of neural network models which use
supervised and unsupervised learning methods, and address problems such as pattern recognition and
prediction.

The primary intuition behind the ART model is that object identification and recognition generally occur as
a result of the interaction of 'top-down' observer expectations with 'bottom-up' sensory information. The
model postulates that 'top-down' expectations take the form of a memory template or prototype that is then
compared with the actual features of an object as detected by the senses. This comparison gives rise to a
measure of category belongingness. As long as this difference between sensation and expectation does not
exceed a set threshold called the 'vigilance parameter', the sensed object will be considered a member of the
expected class. The system thus offers a solution to the 'plasticity/stability' problem, i.e. the problem of
acquiring new knowledge without disrupting existing knowledge that is also called incremental learning.

Learning model
The basic ART system is an unsupervised learning model. It
typically consists of a comparison field and a recognition field
composed of neurons, a vigilance parameter (threshold of
recognition), and a reset module.

The comparison field takes an input vector (a one-

dimensional array of values) and transfers it to its best
match in the recognition field.
Its best match is the single neuron whose set of
Basic ART structure
weights (weight vector) most closely matches the
input vector.
Each recognition field neuron outputs a negative signal (proportional to that neuron's quality
of match to the input vector) to each of the other recognition field neurons and thus inhibits
their output.
In this way the recognition field exhibits lateral inhibition, allowing each neuron in it to
represent a category to which input vectors are classified.

After the input vector is classified, the reset module compares the strength of the recognition
match to the vigilance parameter.
If the vigilance parameter is overcome (i.e. the input vector is within the normal range
seen on previous input vectors), then training commences:
The weights of the winning recognition neuron are adjusted towards the features of
the input vector
Otherwise, if the match level is below the vigilance parameter (i.e. the input vector's
match is outside the normal expected range for that neuron) the winning recognition
neuron is inhibited and a search procedure is carried out.
In this search procedure, recognition neurons are disabled one by one by the reset
function until the vigilance parameter is overcome by a recognition match.
 In particular, at each cycle of the search procedure the most active recognition
neuron is selected and then switched off, if its activation is below the vigilance
parameter
(note that it thus releases the remaining recognition neurons from its inhibition).
If no committed recognition neuron's match overcomes the vigilance parameter, then an
uncommitted neuron is committed and its weights are adjusted towards matching the
input vector.
The vigilance parameter has considerable influence on the system: higher vigilance
produces highly detailed memories (many, fine-grained categories), while lower vigilance
results in more general memories (fewer, more-general categories).

Training
There are two basic methods of training ART-based neural networks: slow and fast. In the slow learning
method, the degree of training of the recognition neuron's weights towards the input vector is calculated to
continuous values with differential equations and is thus dependent on the length of time the input vector is
presented. With fast learning, algebraic equations are used to calculate degree of weight adjustments to be
made, and binary values are used. While fast learning is effective and efficient for a variety of tasks, the
slow learning method is more biologically plausible and can be used with continuous-time networks (i.e.
when the input vector can vary continuously).

Types
ART 1[1][2] is the simplest variety of ART networks, accepting only binary inputs. ART 2[3] extends
network capabilities to support continuous inputs. ART 2-A[4] is a streamlined form of ART-2 with a
drastically accelerated runtime, and with qualitative results being only rarely inferior to the full ART-2
implementation. ART 3[5] builds on ART-2 by simulating rudimentary neurotransmitter regulation of
synaptic activity by incorporating simulated sodium (Na+) and calcium (Ca2+) ion concentrations into the
system's equations, which results in a more physiologically realistic means of partially inhibiting categories
that trigger mismatch resets.

ARTMAP[6] also known as Predictive ART, combines two

slightly modified ART-1 or ART-2 units into a supervised learning
structure where the first unit takes the input data and the second
unit takes the correct output data, then used to make the minimum
possible adjustment of the vigilance parameter in the first unit in
order to make the correct classification.

Fuzzy ART[7] implements fuzzy logic into ART's pattern ARTMAP overview
recognition, thus enhancing generalizability. An optional (and very
useful) feature of fuzzy ART is complement coding, a means of
incorporating the absence of features into pattern classifications, which goes a long way towards preventing
inefficient and unnecessary category proliferation. The applied similarity measures are based on the L1
norm. Fuzzy ART is known to be very sensitive to noise.

Fuzzy ARTMAP[8] is merely ARTMAP using fuzzy ART units, resulting in a corresponding increase in
efficacy.

Simplified Fuzzy ARTMAP (SFAM)[9] constitutes a strongly simplified variant of fuzzy ARTMAP
dedicated to classification tasks.
Gaussian ART[10] and Gaussian ARTMAP[10] use Gaussian activation functions and computations
based on probability theory. Therefore, they have some similarity with Gaussian mixture models. In
comparison to fuzzy ART and fuzzy ARTMAP, they are less sensitive to noise. But the stability of learnt
representations is reduced which may lead to category proliferation in open-ended learning tasks.

Fusion ART and related networks[11][12][13] extend ART and ARTMAP to multiple pattern channels.
They support several learning paradigms, including unsupervised learning, supervised learning and
reinforcement learning.

TopoART[14] combines fuzzy ART with topology learning networks such as the growing neural gas.
Furthermore, it adds a noise reduction mechanism. There are several derived neural networks which extend
TopoART to further learning paradigms.

Hypersphere ART[15] and Hypersphere ARTMAP[15] are closely related to fuzzy ART and fuzzy
ARTMAP, respectively. But as they use a different type of category representation (namely hyperspheres),
they do not require their input to be normalised to the interval [0, 1]. They apply similarity measures based
on the L2 norm.

LAPART[16] The Laterally Primed Adaptive Resonance Theory (LAPART) neural networks couple two
Fuzzy ART algorithms to create a mechanism for making predictions based on learned associations. The
coupling of the two Fuzzy ARTs has a unique stability that allows the system to converge rapidly towards a
clear solution. Additionally, it can perform logical inference and supervised learning similar to fuzzy
ARTMAP.

Criticism
It has been noted that results of Fuzzy ART and ART 1 (i.e., the learnt categories) depend critically upon
the order in which the training data are processed. The effect can be reduced to some extent by using a
slower learning rate, but is present regardless of the size of the input data set. Hence Fuzzy ART and ART
1 estimates do not possess the statistical property of consistency.[17] This problem can be considered as a
side effect of the respective mechanisms ensuring stable learning in both networks.

More advanced ART networks such as TopoART and Hypersphere TopoART that summarise categories to
clusters may solve this problem as the shapes of the clusters do not depend on the order of creation of the
associated categories. (cf. Fig. 3(g, h) and Fig. 4 of [18])

References
1. Carpenter, G.A. & Grossberg, S. (2003), Adaptive Resonance Theory (http://cns.bu.edu/Profil
es/Grossberg/CarGro2003HBTNN2.pdf) Archived (https://web.archive.org/web/2006051909
1948/http://cns.bu.edu/Profiles/Grossberg/CarGro2003HBTNN2.pdf) 2006-05-19 at the
Wayback Machine, In Michael A. Arbib (Ed.), The Handbook of Brain Theory and Neural
Networks, Second Edition (pp. 87-90). Cambridge, MA: MIT Press
2. Grossberg, S. (1987), Competitive learning: From interactive activation to adaptive
resonance (http://www.cns.bu.edu/Profiles/Grossberg/Gro1987CogSci.pdf) Archived (https://
web.archive.org/web/20060907041202/http://www.cns.bu.edu/Profiles/Grossberg/Gro1987C
ogSci.pdf) 2006-09-07 at the Wayback Machine, Cognitive Science (journal), 11, 23-63
 3. Carpenter, G.A. & Grossberg, S. (1987), ART 2: Self-organization of stable category
recognition codes for analog input patterns (http://cns-web.bu.edu/Profiles/Grossberg/CarGr
o1987AppliedOptics.pdf) Archived (https://web.archive.org/web/20060904212143/http://cns-
web.bu.edu/Profiles/Grossberg/CarGro1987AppliedOptics.pdf) 2006-09-04 at the Wayback
Machine, Applied Optics, 26(23), 4919-4930
4. Carpenter, G.A., Grossberg, S., & Rosen, D.B. (1991a), ART 2-A: An adaptive resonance
algorithm for rapid category learning and recognition (http://cns.bu.edu/Profiles/Grossberg/C
arGroRos1991NNART2A.pdf) Archived (https://web.archive.org/web/20060519092850/htt
p://cns.bu.edu/Profiles/Grossberg/CarGroRos1991NNART2A.pdf) 2006-05-19 at the
Wayback Machine, Neural Networks, 4, 493-504
5. Carpenter, G.A. & Grossberg, S. (1990), ART 3: Hierarchical search using chemical
transmitters in self-organizing pattern recognition architectures (http://cns.bu.edu/Profiles/Gr
ossberg/CarGro1990NN.pdf) Archived (https://web.archive.org/web/20060906014656/http://
cns.bu.edu/Profiles/Grossberg/CarGro1990NN.pdf) 2006-09-06 at the Wayback Machine,
Neural Networks, 3, 129-152
6. Carpenter, G.A., Grossberg, S., & Reynolds, J.H. (1991), ARTMAP: Supervised real-time
learning and classification of nonstationary data by a self-organizing neural network (http://c
ns.bu.edu/Profiles/Grossberg/CarGroRey1991NN.pdf) Archived (https://web.archive.org/we
b/20060519091848/http://cns.bu.edu/Profiles/Grossberg/CarGroRey1991NN.pdf) 2006-05-
19 at the Wayback Machine, Neural Networks, 4, 565-588
7. Carpenter, G.A., Grossberg, S., & Rosen, D.B. (1991b), Fuzzy ART: Fast stable learning and
categorization of analog patterns by an adaptive resonance system (http://cns.bu.edu/Profile
s/Grossberg/CarGroRos1991NNFuzzyART.pdf) Archived (https://web.archive.org/web/2006
0519091505/http://cns.bu.edu/Profiles/Grossberg/CarGroRos1991NNFuzzyART.pdf) 2006-
05-19 at the Wayback Machine, Neural Networks, 4, 759-771
8. Carpenter, G.A., Grossberg, S., Markuzon, N., Reynolds, J.H., & Rosen, D.B. (1992), Fuzzy
ARTMAP: A neural network architecture for incremental supervised learning of analog
multidimensional maps (http://cns.bu.edu/Profiles/Grossberg/CarGroMarRey1992IEEETrans
NN.pdf) Archived (https://web.archive.org/web/20060519094345/http://cns.bu.edu/Profiles/G
rossberg/CarGroMarRey1992IEEETransNN.pdf) 2006-05-19 at the Wayback Machine, IEEE
Transactions on Neural Networks, 3, 698-713
9. Mohammad-Taghi Vakil-Baghmisheh and Nikola Pavešić. (2003) A Fast Simplified Fuzzy
ARTMAP Network, Neural Processing Letters, 17(3):273–316
10. James R. Williamson. (1996), Gaussian ARTMAP: A Neural Network for Fast Incremental
Learning of Noisy Multidimensional Maps (http://dcommon.bu.edu/bitstream/handle/2144/21
80/95.003.pdf?sequence=1&isAllowed=y), Neural Networks, 9(5):881-897
11. Y.R. Asfour, G.A. Carpenter, S. Grossberg, and G.W. Lesher. (1993) Fusion ARTMAP: an
adaptive fuzzy network for multi-channel classification (http://open.bu.edu/xmlui/bitstream/ha
ndle/2144/2029/93.052.pdf?...1&sa=U&ei=ZpBPU5iSAZCWyAToioHIDg&ved=0CC4QFjAE
&usg=AFQjCNGbPmXW27YRaMlcQyD9FQLDNXB7lg). In: Proceedings of the Third
International Conference on Industrial Fuzzy Control and Intelligent Systems (IFIS).
12. Tan, A.-H.; Carpenter, G. A.; Grossberg, S. (2007). Liu, D.; Fei, S.; Hou, Z.-G.; Zhang, H.;
Sun, C. (eds.). "Intelligence Through Interaction: Towards a Unified Theory for Learning" (htt
ps://link.springer.com/chapter/10.1007%2F978-3-540-72383-7_128). Advances in Neural
Networks – ISNN 2007. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer.
4491: 1094–1103. doi:10.1007/978-3-540-72383-7_128 (https://doi.org/10.1007%2F978-3-5
40-72383-7_128). ISBN 978-3-540-72383-7.
13. Tan, A.-H.; Subagdja, B.; Wang, D.; Meng, L. (2019). "Self-organizing neural networks for
universal learning and multimodal memory encoding" (https://linkinghub.elsevier.com/retriev
e/pii/S0893608019302370). Neural Networks. 120: 58–73.
doi:10.1016/j.neunet.2019.08.020 (https://doi.org/10.1016%2Fj.neunet.2019.08.020).
PMID 31537437 (https://pubmed.ncbi.nlm.nih.gov/31537437). S2CID 202703163 (https://ap
i.semanticscholar.org/CorpusID:202703163).
14. Marko Tscherepanow. (2010) TopoART: A Topology Learning Hierarchical ART Network (htt
ps://pub.uni-bielefeld.de/download/1925596/2499061), In: Proceedings of the International
Conference on Artificial Neural Networks (ICANN), Part III, LNCS 6354, 157-167
15. Georgios C. Anagnostopoulos and Michael Georgiopoulos. (2000), Hypersphere ART and
ARTMAP for Unsupervised and Supervised Incremental Learning (http://techlab.bu.edu/file
s/resources/articles_tt/Anagnostopoulos_Georgiopoulos_2000.pdf), In: Proceedings of the
International Joint Conference on Neural Networks (IJCNN), vol. 6, 59-64
16. Sandia National Laboratories (2017) Lapart-python documentation (https://lapart-python.rea
dthedocs.io/en/latest/index.html#lapart-python-documentation)
17. Sarle, Warren S. (1995), Why Statisticians Should Not FART (http://medusa.sdsu.edu/Roboti
cs/Neuromuscular/Articles/ATM_articles/fart.txt) Archived (https://web.archive.org/web/20110
720042616/http://medusa.sdsu.edu/Robotics/Neuromuscular/Articles/ATM_articles/fart.txt)
July 20, 2011, at the Wayback Machine
18. Marko Tscherepanow. (2012) Incremental On-line Clustering with a Topology-Learning
Hierarchical ART Neural Network Using Hyperspherical Categories (https://pub.uni-bielefel
d.de/download/2498997/2517690/tscherepanow.marko2012incremental-ICDM.pdf), In:
Poster and Industry Proceedings of the Industrial Conference on Data Mining (ICDM), 22–34

Wasserman, Philip D. (1989), Neural computing: theory and practice, New York: Van Nostrand Reinhold,
ISBN 0-442-20743-3

External links
Stephen Grossberg's website (https://web.archive.org/web/20060616203130/http://cns-web.
bu.edu/Profiles/Grossberg/)
ART's implementation for unsupervised learning (ART 1, ART 2A, ART 2A-C and ART
distance) (https://web.archive.org/web/20120109162743/http://users.visualserver.org/xhudik/
art)
Summary of the ART algorithm (https://web.archive.org/web/20160908192735/http://www.cn
s.bu.edu/Profiles/Grossberg/ART.pdf)
LibTopoART (https://www.LibTopoART.eu) — TopoART implementations for supervised and
unsupervised learning (TopoART, TopoART-AM, TopoART-C, TopoART-R, Episodic
TopoART, Hypersphere TopoART, and Hypersphere TopoART-C)

Retrieved from "https://en.wikipedia.org/w/index.php?title=Adaptive_resonance_theory&oldid=1131192463"