Chemometrics and

intelligent
laboratory systems
Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329

Adaptive resonance theory based artificial neural networks for

treatment of open-category problems in chemical pattern
recognition - application to UV-Vis and IR spectroscopy
Dietrich Wienke *, Gerrit Kateman
Catholic University of Nijmegen, Laboratory of Analytical Chemistry, Toernooiveld 1, 6525 ED Nijmegen, Netherlands

(Received 11 January 1993; accepted 26 September 1993)

Abstract

A supervised classification problem is defined as open-categorical if the training classes are only partly known
beforehand according to their number, shape and size. This is the case, for example, in applications of spectroscopic
pattern recognition for automated materials sorting for recycling. Continuously modified or newly developed
materials require a continuous self-adapting pattern recognition technique (plasticity). A second, but contrary
demand is the robustness of the classifier against outlier pattern vectors (stability). This stability-plasticity dilemma
can partly be solved using Grossberg’s adaptive resonance theory based artificial neural networks (ARTS). The main
difference between ARTS like artificial neural networks to other types of artificial neural networks is that besides
the numerical size of the weights as fitting parameters, additionally the network structure itself (number of units,
dimensions of the weights matrices) is also not fixed forming a final result of the training process. Basic properties of
the ART-l technique for its potential application to chemical pattern recognition are elucidated by a classification of
simulated data, UV-Vis spectra of phenanthroline complexes and infrared reflectance spectra of optical glasses.

1. Introduction another example realize some principles of natu-

ral neural networks such as parallel information
Many chemometrically oriented papers were processing and distributed knowledge storage and
published during the last years dealing with retrieval. After a first cross reference of Jurs et
mathematical techniques of natural computing al. in 1969 [3] in connection with the classification
for applications in analytical chemistry and spec- of mass spectra, in 1975 Stonham et al. 141used
troscopy. The genetic algorithm as one example an ‘adaptive digital learning network’ to classify
for natural computing methods exploits the prin- mass spectra. Influenced by the introduction of
ciple of natural evolution for chemical-analytical the backpropagation learning rule in the begin-
applications [ 1,2]. Artificial neural networks as ning of the eighties analytical chemists and spec-
troscopists started to deal more intensively with
potential applications of neural networks. Thom-
son and Meyer 151and Anker and Jurs 161classi-
* Corresponding author. fied NMR spectra by means of multilayer feed-

Elsevier Science B.V.

SSDI 0169-7439(93)E0063-A
310 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329
forward neural networks based on the backpropa-
gation learning rule (MLF). Bos et al. 171 classi-
fied with the same network type data from ion-
selective electrode arrays. Nakamato et al. [8] and
Chay et al. [9] analysed data from multi-sensor
arrays for the differentiation of drinks and for
detection of odorants in general. Bos and Weber
[lo], Glick and Hieftje [ll] and Otto et al. [12]
reported applications in different atomic spectro-
scopic methods such as X-ray fluorescence, glow
discharge and inductively coupled plasma (ICP)
spectroscopy. Schierle et al. [13] used in place of
a MLF network the bidirectional associative
memory (BAM) for the qualitative interpretation
of ICP spectra. The interpretation of infrared
spectra by MLF neural networks was reported by
Robb and Munk [14], Wythoff et al. [15], Smits et
al. [16], Meyer and Weigelt [17] and Tanabe et al.
[US].Nonlinear multivariate calibration with MLF
networks in the field of near-infrared spec-
troscopy was discussed by Long and co-workers
[19,20]. Lang et al. [21] applied MLF networks to
analytical chromatography. An application of the
MLF neural networks to the interpretation of
mass spectra was published by Lohninger [22]. In
a review article Zupan and Gasteiger [23] pre-
sented additional chemical applications of neural
networks outside analytical chemistry, for exam-
ple, in chemical product soft modeling and chem-
ical process control. In an overview article Jans-
son [24] focussed on potential application of a
third type of artificial neural networks, the Hop-
field type neural networks, in analytical chem-
istry. A fourth type of artificial neural networks,
the unsupervised working Kohonen self-organiz-
ing features map, was used by Gross and Seibert
[25] to evaluate satellite images of remote spec-
trochemical measurements of environmental data.
They mapped the images by Kohonen’s method
and used the detected qualitative clusters to form
representative training classes for backpropaga-
tion neural networks and supervised pattern
recognition with the vector quantizer method.
Arrigo et al. [26] used the method to evaluate
nucleic acid sequences. An application to a quan-
titative structure-activity relationship (QSAR)
study has been reported recently by Ross et al.
[27]. Melssen et al. [28] clustered a data base of
IR spectra by a parallel implementation of the
network.
These four types of artificial neural networks
used in analytical chemistry (MLF backpropaga-
tion network, BAM, Hopfield’s neural network
and Kohonen self-organizing feature map) work
with a predefined rigid network design. This
means that the number of layers, the number of
units and the number of weights are fixed from
the start of the training process up to the conver-
gence of the network. The numerical size of the
weights is the only flexible parameter which can
be used for storage and retrieval of information.
Or, in other words, the structure itself of the
neural network is not a free parameter during the
learning process. A practical way out is the repe-
tition of the training with the same data but
different network designs or the rejection of cer-
tain less important weights as reported by Har-
rington [29] and by design of optimal minimal
neural networks as discussed by Borggaard and
Thodberg [30].
However, a rigid network structure can be a
deficiency under certain circumstances. A
changed classification environment with a strong
deviating classification task, for example, can in-
fluence the learned and stored knowledge in an
undesirable direction. In such a situation the
network provides uninterpretable results. Gross-
berg [31-341 called this problem the stability-
plasticity dilemma.
1.2. Stability-plasticity dilemma of artificial neural
networks
Grossberg [31, p. 851 stated how a system’s
adaptive mechanism can be stable enough to re-
sist environmental fluctuations, but plastic enough
to change rapidly in response to environmental
demands. Further he asked how stability without
rigidity and adaptability without chaos could be
achieved. Also in ref. [31, p. 1893 the aspects of
the stability-plasticity dilemma are described,
how internal representations can maintain them-
selves in a stable fashion against the erosive ef-
fects of behaviorally irrelevant environmental
fluctuations yet can nonetheless adapt rapidly in
response to environmental fluctuations that are
 D. henke, G. Kateman /Chemometrics
crucial to survival. Grossberg searched for a solu-
tion that a network knows the difference between
behaviorally irrelevant and relevant events even
though its individual cells, or nodes, do not pos-
sess this knowledge. For the practical realization
of such a new type of learning it is further impor-
tant how a network is able to transmute this
knowledge into the difference between slow and
fast rates of adaptation, respectively. Based on
biological, neurological and mathematical studies
he developed an artificial neural network to over-
come the stability-plasticity dilemma. The new
network differs significantly from other known
network types [35]. It is based on adaptive reso-
nance theory (ART) which will be described in
more detail in the theoretical part. The use of
ART based neural networks has been reported
outside analytical chemistry for classification of
simulated data by Carpenter and Grossberg [36-
421, for clustering of large data sets by Burke 1431
and for on-line recognition of hand-written let-
ters by Gan [44]. Further citations referring to
applications of modified ART algorithms in im-
age processing, speech recognition and sonar sig-
nal interpretation can be found in Carpenter and
Grossberg [42].
The present work deals with properties and
behavior of the ART-l algorithm in a computer
simulation study and in two chemical-analytical
application cases for the classification of UV-Vis
spectra of coloured phenanthroline complexes
and for the classification of Fourier transform
infrared reflectance spectra of glasses with differ-
ent compositions. The aim of this first study is to
introduce basic ideas of ART for chemists.
2. Adaptive resonance theory (ART)
A general design of an artificial neural net-
work based on adaptive resonance theory (ART)
shows at least one input layer, two working layers
1 and 2 and one output layer. The layers 1 and 2
are connected with each other in a forward mode
by a weight matrix Wl and in a feedback mode by
a weight matrix W2. The number of units in layer
1 corresponds to the number, m, of variables
and Intelligent Laboratory Systems 23 (1994) 309-329 311
(features) in the input vector. The number of
units in layer 2 is unlimited, in principle. How-
ever, in practice the actual number of units in
layer 2 is determined by the actually detectable
number, k, of clusters in the input data. The
input data vector is transferred from the outside
world via the input layer and a transfer function
fl into layer 1. After processing in the network a
corresponding output vector in layer 2 is ob-
tained. This vector will be sent via another trans-
fer function f2 to an output layer.
General desired properties for all kinds of
artificial neural networks are robustness against
events from outside, stability of recognition re-
sults in a new classification environment and the
self-design of the network layout. Grossberg [371
was able to realize simultaneously these three
properties within his ART based neural network
by introducing an unique operation called ‘feed-
back by weights’. The output of layer 2 obtained
from an input vector is sent back via the weight
matrix W2 (top-down weights) and transfer func-
tion fl to the layer 1. This feedback operation
occurs before an output vector is sent via the
output layer to the outside world. This feedback
causes every new input coming from outside to
propagate two times through the neural net: one
time through the ‘bottom-up weights’ Wl and
one time via feedback through W2. The network
tries to adjust the two weight matrices Wl and
W2 in such a way that the original input vector
and the, by means of Wl and W2, ‘filtered’ input
vector match each other. To reach the feedback
behavior Grossberg linked two neural network
elements called ‘instar’ and ‘outstar’. The instar
network is a vector detector and operates with a
list of neurons which has an ‘unlimited’ length, in
principle. If the first input vector is trained, a
first neuron from this list is set equal to one. If a
new training vector is offered which differs signif-
icantly from the first one a new second neuron is
activated in the list and in this way the dimension
of the weight matrix is enlarged. The first neuron
and all other potential neurons are set to zero.
Thus, the instar generates a vector of limited
length, k, of neurons actually in use where one
neuron is set active (‘1’) and the others are set
non-active (‘0’). The instar uses for stepwise
312 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329
adaptation of the j xi weight matrix Wl the
learning rule
(1)
with 77= learning rate, xi = input vector of di-
mension 1 x m with j = l... m, a(i) = output
vector of dimension 1 x k with i = 1.. . k, m =
number of input features, k = number of output
neurons actually in use corresponding to the
number of classes.
An outstar artificial neural network does the
opposite compared with an instar network. A list
of k neurons is presented to the outstar network
were the ith neuron is firing (‘1’) and the others
are set to non-active (‘0’). Then from the outstar
an m-dimensional output vector is obtained, for
example, a continuous spectrum corresponding to
the class i assigned by the firing outstar neuron.
For updating the i X j weight matrix W2 the
outstar uses the following learning rule
dWZi,j=?) * [t%(i)- ’ (k + 0th output neuron is needed or not is based
w2i,j] * xj
(2)
with xj = output vector of dimension 1 x m with
j= l... m, a(i) = input vector of dimension 1 x k
with i= l...k.
By a link of instar and outstar neural networks
Grossberg created a feedback cycle as a basis for
the self-stabilizing properties of such network
types.
If a test vector xj, which does not differ from
the training vectors, is presented to such a com-
pletely trained neural network it will stimulate a
neuron in the second layer to fire. This firing
neuron in the outstar stimulates via W2 the for-
mation of a vector x,? in layer 1 which will be
identical to the original test vector xj. The trained
neural network comes into perfect resonance with
the test vector presented to the input layer.
If a slightly differing test vector is presented to
such a trained network that is not significantly
different from the training vectors then the net-
work at first slightly adapts its actual weights
using Eqs. (1) and (2). The network shows a
certain plasticity. Secondly, resonance occurs
again between the slightly deviating test vector
and the trained and new adapted neural network.
That is why Grossberg has introduced for his new
type of neural networks with feedback weights
the name ‘adaptive resonance theory based artifi-
cial neural networks’ (ART).
A test vector xj not belonging to any previous
trained class will not stimulate resonance be-
tween network and test input. On the other hand,
such a strong deviating vector will not destroy the
previously stored knowledge (network stability).
The significantly deviating test vector at first
stimulates the enlargement of the layer 2 by link-
ing one additional new output neuron to it and
enlarging in this way the dimension k of the
weight matrices Wl and W2 by one.
2.1. Competitive learning and the novelty detector
The self formation of classes requires a contin-
uous update of the dimension k of the weight
matrices Wl and W2. The number k of i = 1.. . k
output units in layer 2 increases with the number
of detected clusters. The decision if an additional
on a decision coming from a so-called novelty
detector. It works as follows. Every new input
vector xj stimulates via its dot product with Wl
simultaneous activities of all k output neurons in
layer 2. Every excited ith output neuron stimu-
lates via W2 the formation of a vector x:(i) in
layer 1. Based on vector similarity between the
ith xj* and the xr a winning neuron in layer 2 can
be selected. A high vector similari~ is mathemat-
ically expressed by a small vector angle B
ej(i)[xj, xi*(i)]
=arcoss[x~xI*(i)]/[~~~~/~*~(x~(i)~~] (3)
At the end, one remaining ith output neuron
in layer 2 will dominate the competition within
the feedback cycle. Its corresponding vector xj(i)
will be compared with the input vector X~ using a
vigilance parameter p with
Pj(i) =XjTXj*(i)/llXjll (4)
which looks similar to the vector angle in Eq. (3).
A marginal vigilance parameter prnax determines
how much variance within every class box may be
tolerated, in principle. If pj(i) is smaller than
P max then xi is ‘in resonance’ with x,?(i). In the
following calculation step the weights in Wl and
 D. Wienke, G. Kateman / Chemometrics
W2 corresponding to the ith output neuron will
be adapted using Eqs. (1) and (2). This adapta-
tion reinforces the activity of the ith neuron in
layer 2 compared with its activity after a first
initialization. In this way its relative contrast to
other active neurons is enhanced. Sirnultane-
ously, the activity of these other neurons is low-
ered in a future competition if a new input pat-
tern is presented that resonates again with the
ith neuron. The completed process is called com-
petitive feedback learning.
If pi(i) exceeds prnaxthe network is not able to
classify the input vector in any previously formed
class box. The network itself concludes that it has
detected a ‘novelty’. Thus, the network decides to
create an additional output neuron k + 1 in layer
2. It increases the dimension of Wl and W2 by
one and adapts the additional weights immedi-
ately to the deviating input vector using Eqs. (1)
and (2).
2.2. Gain control
If, for example, the outstar neural network in
layer 2 is stimulated independently, it generates a
vector x,? in layer 1. This is again a new input
stimulus for layer 2. This can be an undesired
source for cyclic vibrations between layers 1 and 2
without any reliable input. To overcome such
potential ‘ghost activities’, a bias is used as gain
control. The gain control only opens the gate for
cyclic information flow between layer 1 and layer
2 after a new vector has been presented to the
input layer.
2.3. Transfer functions
The central role of the type of the transfer
functions for the effect of a single neuron was
considered in numerous papers and books. In this
paper we focus first on the ART-l artificial neu-
ral networks with binary scaled input data. Such
an ART-l based neural network uses in layer 1 a
hard limiter transfer function f1 with
(5)
and Intelligent Laboratory Systems 23 (1994) 309-329 313
and in layer 2 the linear increasing transfer func-
tion
with 6, bias constant and b, amplitude constant.
2.4. Other ART algorithms
Carpenter and Grossberg [36,37,39,41,42] de-
veloped a series of different ART algorithms.
The main difference between ART-l 1371 and
ART-2 1361and ART-2a [41] is that both ART-2
algorithms are able to process not only binary
coded but continuous data types. The ART-3
algorithm [39] uses in place of layer 1 and layer 2
additional hidden layers for the treatment of more
complicate cluster structures. FUZZY-ART [42]
can learn binary as well as continuous coded
feature vectors. ART-MAP [45] has been devel-
oped recently to project two feature spaces i and
j onto each other. In this paper we restrict our-
selves to the study of the properties of the ART-l
strategy for its potential use in the pattern recog-
nition of optical spectra. The study of other ART
algorithms will be part of future work.
3. Experimental
3.1. Experimental 1. W-l/is spectroscopy of chem-
ical complexes
The classical phenanthroline method [461 is a
technique for the photometric determination of
Fe in the ppm and in the sub-ppm concentration
range. Fe forms an intensively yellow coloured
Fe-phenanthroline complex (Fe-Phen) having a
spectrum with a maximum absorbance between
480 and 550 nm. Wienke [471 has shown recently
that Co forms with phenanthroline an intensively
orange coloured complex (Co-Phen). Its maxi-
mum UV-Vis absorbance near 220 nm continu-
ously decreases to the noise level near 650 nm in
contrast to the Fe-Phen spectrum. Examples of
the complete absorbance spectra of Fe-Phen and
Co-Phen between 380 and 670 nm were given in
ref. [47].
314 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329

Twelve parallel samples of binary aqueous
mixtures containing 30 kg Co and 3 c1.gFe were
prepared according to ref. [46]. The 12 ab-
sorbance spectra were recorded in the optical
range between 380 and 670 nm and numerically
sampled at 13 equidistant wavenumbers in steps
of 500 cm-’ (computer driven spectrophotometer
M42, Carl Zeiss, Germany) (Table 1, MIXl-12).
Twelve aqueous solutions containing only 3.0 Fg
Fe were prepared independently using the same
phenanthroline method [46]. In the same way 12
aqueous solutions containing only 30 Fg Co were
independently prepared. From the 24 pure solu-
tions absorbance spectra were recorded in the
same optical range sampled at the same 13
wavenumbers. By use of a quartz cuvette with 2.0
cm thickness the spectra of the Co-Phen and
Fe-Phen solutions reached absorbance values be-
tween 0.3 and 0.8 in the chosen optical compro-
mise range. After that the 12 pure spectra of
Co-Phen and of Fe-Phen were ranked within
their group in a random order. Then two corre-
sponding spectra from every set were added at
every wavenumber (Table 1, LINC014-24 from
‘linear combination’).
Thus, for the pattern recognition study with
the ART-l neural network two classes (MIX,
LINGO) individually containing 12 experimental
UV-Vis spectra sampled at 13 wavenumbers were
available (Table 1).
3.2. Experimental 2. Infrared reflectance spec-
troscopy of ternary Al,O,-CaO-SO, glasses
In the study, 66 differently composed glasses
were included. The simultaneously changed con-

Table 1
In the pattern recognition study used 24 experimentally measured UV-Vis absorbance spectra of mixtures of Fe-phenanthroline
and Co-phenanthroline in equally concentrated aqueous solutions containing 30 ug Fe and 3 ug Co. MIXl-12 are spectra of 12
chemically prepared binary mixtures of Fe and Co. Spectra LINC012-24 are pairwise linear combinations of 12 separately
prepared pure solutions of Co-phenanthroline and Fe-phenanthroline (mathematically prepared mixture spectra, see text)
Sample name Wavenumber/cm-’
and category
19000 19500 20000 20500 21000 21500 22000 22500 23000 23500 24000 24500 25000
MIX1 1.070 1.166 1.13 1.13 1.116 1.07 1.021 0.98 0.94 0.898 0.842 0.818 0.790
MIX2 1.075 1.176 1.17 1.145 1.121 1.09 1.028 0.998 0.945 0.901 0.854 0.828 0.802
MIX3 1.054 1.17 1.11 1.115 1.10 1.06 0.99 0.962 0.919 0.878 0.820 0.796 0.764
MIX4 1.075 1.178 1.152 1.13 1.136 1.07 1.001 0.975 0.936 0.886 0.835 0.813 0.770
MIX5 1.075 1.21 1.17 1.142 1.119 1.09 1.008 0.98 0.925 0.881 0.832 0.799 0.772
MIX6 1.075 1.22 1.18 1.15 1.14 1.096 1.021 0.99 0.935 0.887 0.838 0.802 0.784
MIX7 1.04 1.188 1.172 1.16 1.145 1.09 1.021 0.97 0.935 0.890 0.840 0.792 0.778
MIX8 1.055 1.20 1.204 1.172 1.155 1.112 1.055 0.982 0.948 0.905 0.857 0.825 0.784
MIX9 1.06 1.193 1.158 1.152 1.137 1.082 1.024 0.986 0.948 0.888 0.855 0.814 0.788
MIX10 1.065 1.202 1.19 1.156 1.155 1.11 1.04 1.012 0.957 0.908 0.860 0.816 0.794
MIX11 1.04 1.17 1.143 1.128 1.105 1.08 1.Ol 0.982 0.945 0.906 0.865 0.822 0.803
MIX12 1.045 1.185 1.16 1.135 1.11 1.09 1.018 0.988 0.95 0.910 0.867 0.827 0.811

LINC013 1.038 1.181 1.147 1.15 1.132 1.11 1.022 0.968 0.916 0.872 0.806 0.756 0.695
LINC014 1.052 1.188 1.173 1.18 1.166 1.121 1.007 0.992 0.931 0.883 0.812 0.768 0.706
LINCOlS 1.048 1.194 1.174 1.155 1.139 1.086 1.022 0.98 0.933 0.873 0.826 0.744 0.736
LINC016 1.062 1.218 1.192 1.175 1.159 1.105 1.035 0.998 0.942 0.890 0.835 0.755 0.747
LINC017 1.051 1.221 1.194 1.162 1.16 1.107 1.02 0.995 0.938 0.884 0.814 0.768 0.718
LINC018 1.073 1.235 1.214 1.178 1.156 1.113 1.032 1.005 0.961 0.898 0.826 0.778 0.727
LINC019 1.052 1.186 1.154 1.16 1.147 1.09 1.017 0.987 0.936 0.860 0.805 0.759 0.714
LINC020 1.075 1.227 1.186 1.188 1.169 1.122 1.025 0.996 0.994 0.887 0.816 0.765 0.728
LINC021 1.068 1.205 1.157 1.15 1.148 1.105 1.02 0.989 0.931 0.880 0.816 0.771 0.718
LINC022 1.072 1.225 1.188 1.161 1.16 1.093 1.037 0.997 0.938 0.886 0.828 0.773 0.740
LINC023 1.053 1.186 1.177 1.138 1.13 1.092 1.005 0.967 0.938 0.854 0.81 0.781 0.697
LINC024 1.063 1.211 1.193 1.149 1.142 1.12 1.088 0.986 0.938 0.873 0.825 0.764 0.718
 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329 315

tents of the three oxides ranged between Al,O,
= 0.00-36.67 m%, CaO = 10.00-55.0 m% and
SiO, = 26.89-80.00 m%. After careful polishing
of the sample surface every glass was character-
ized by its Fourier transform infrared reflectance
(FT-IRR) spectrum in the range 400-1600 cm-’
with a resolution of 1.9285 cm-’ under a con-
stant angle of excitation of 57”. Every sample
spectrum was obtained from an accumulation of
50 repeated scans over the chosen optical range.
Further details of the used standard procedure of
ET-IRR measurements at glass surfaces can be
found in ref. [48]. The 66 raw spectra were re-
duced to the optical range 400-1200 cm-‘. In
this wavenumber region the most important vi-
brations of the ternary Al,O,-CaO-SiO, glasses
can be observed 149,501. The raw high resolution
spectra were equidistantly sampled at only 69
wavenumbers to reduce the amount of data and
because of the low number of bands and their
broad distributions over the considered spectral
range.
Thus, a matrix of 66 spectral pattern vectors
described by 69 features formed the input data
set for the second experimental ART-l study.
3.3. Computational
The computer program was written by the
authors in MATLAB [511 with partial use of the
artone1.m routine from the MATLAB Neural
Networks Toolbox from Demuth and Beale [52].
The program concerns visualization of the raw
experimental spectra, graphical two-dimensional
principal components mapping, feature autoscal-
ing and feature range scaling, feature binary cod-
ing, unsupervised ART-l learning, supervised
ART-l classification and graphical visualization
of the weight matrices.
4. Results and discussion
4.1. A computer simulation study
Eight simulated training spectra and three
simulated test spectra (Table 2) were used. A
visual inspection of the 11 simulated spectra (Ta-
ble 2, Fig. 1) shows that the training spectra
LEARNl-4 (Fig. la) form one group and that
the training spectra LEARNS-8 (Fig. la) form a
second group with a certain variation within every
single group. The test spectrum TEST1 (Fig. lb)
seems to belong to the group LEARNl-4, the
test spectrum TEST2 seems to belong to
LEARNS-8 and the test spectrum TEST3 seems
to form a third cluster well separated from the
other spectra. An artificial neural network was
designed with 13 input units corresponding to the
m = 13 input variables (wavelength intervals) of

Table 2
Computer simulated absorbance spectra sampled at 13 equidistant wavelengths. Two groups LEARNl-4 and LEARNS-8 of in
total eight training spectra and three test spectra TESTl-3 were generated numerically
Name of spectrum Simulated absorbance at wavelenath/nm
- I

220 240 260 280 300 320 340 360 380 400 420 460 480

LEARN1 0.7 1.0 0.8 0.7 0.6 0.6 0.4 0.4 0.4 0.3 0.1 0.1 0.0
LEARN2 0.9 1.0 0.9 0.8 0.6 0.6 0.6 0.5 0.4 0.2 0.0 0.0 0.0
LEARN3 0.6 0.7 1.0 0.9 0.7 0.7 0.7 0.5 0.4 0.4 0.4 0.2 0.0
LEARN4 0.7 0.9 1.0 0.8 0.6 0.5 0.4 0.4 0.1 0.1 0.0 0.0 0.2

LEARN5 0.2 0.1 0.4 0.6 0.7 0.7 0.6 0.6 0.8 0.9 1.0 0.9 0.9
LEARN6 0.1 0.3 0.4 0.4 0.6 0.5 0.6 0.7 0.7 0.9 1.0 0.8 0.7
LEARN7 0.0 0.1 0.1 0.4 0.6 0.6 0.6 0.7 0.8 0.9 0.9 0.9 1.0
LEARN8 0.1 0.2 0.4 0.5 0.8 0.7 0.6 0.7 0.8 1.0 0.8 0.7 0.6

TEST1 0.7 0.9 0.9 0.8 0.6 0.6 0.5 0.5 0.3 0.2 0.1 0.1 0.0
TEST2 0.1 0.2 0.4 0.4 0.5 0.6 0.6 0.7 0.7 0.9 0.9 0.8 0.8
TEST3 0.1 0.2 0.3 0.5 0.7 0.9 1.0 1.0 0.9 0.7 0.2 0.1 0.1
316 D. Wienke, G. Kateman / Chemometrics
the it = 11 pattern vectors (spectra). A pool of 20
output units in layer 2 was chosen. The ab-
sorbances of the original spectra from Table 1
were binary coded between 0 and 1. One possibil-
ity for binary coding of continuous data will be
given in detail in the next chapter.
Five computer experiments were carried out
by using these 11 binary coded spectra. In every
single computer experiment the ART-1 neural
network was at first trained with the eight train-
ing spectra LEARNl-8. Then the trained neural
network was used for a reclassification of the
training spectra LEARNl-8 (recognition step)
and for classification of the test spectra TESTl-3
(prediction step). The only difference between all
five experiments was an increasing size of the
vigilance parameter p chosen in the steps of
p = 0.2, 0.7, 0.78, 0.80, 0.90.
From the first classification results (Table 3,
columns 1 and 2) can be seen that the ART
network decides to form two output units corre-
sponding to two detected clusters in the training
step for p = 0.2. In the test step for p = 0.2 the
network classifies the spectra TEST1 and TEST2
into these two formed and trained class boxes.
However, the spectrum TEST3 belongs neither to
the class 1 nor to the class 2. The network de-
cides to create an additional third output unit.
Thus, the network has detected two clusters in
the training and in the test spectra and one third
cluster in the test set. This corresponds to the
and Intelligent Laboratory Systems 23 (1994) 309-329
visual impression obtained from the spectra plots
in Fig. 1. Looking further to the results in Table 2
it is obvious that a certain increase in the vigi-
lance parameter from 0.2 up to 0.9 is followed by
the formation of more subclusters within both
groups of spectra. Anyway, after formation of the
class boxes the network is able to classify the test
spectra TEST1 and TEST2 into the correct sub-
groups. The third test spectrum TEST3 stimu-
lates in every case the formation of a distinguish-
ing new class box.
4.2. Experimental 1. W-Vii spectra of phenan-
throline complexes
4.2.1. Raw spectra
A score plot of the 24 spectra obtained by
principal component analysis (PCA) of the au-
toscaled 24 x 13 absorbance matrix from Table 1
shows a complete separation of the 12 spectra of
the MIX group from the 12 spectra of the LINCO
group in the direction of the second principal
component (Fig. 2). Wienke [47] described as
reason for the clear two-group separation of the
24 spectra a disturbed additivity in the wavenum-
ber range between 23 500 and 25 000 cm-
 D. Wienke, G. Ka!eman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329 317

4.2.2. Binary coding of the spectra 13 absorbance matrix of the 24 experimental

For binary coding of the 24 spectra their 13 UV-Vis spectra (Table 4). To prevent that any
features were autoscaled with the total mean and spectrum gets only zeros as features, the zeros
the total standard deviation over all 24 spectra and ones for wavenumber 25 000 cm- ’
for every single wavenumber. Additionally the 13
features were range scaled between 0 and + 1.
Because of outliers and deviations from normal
distribution the median as robust mean value [56]
was used as a decision threshold for a binary
coding of the between 0 and 1 scaled ab-
sorbances. This resulted in the binary coded 24 X

ABSORBANCE (simulated)

18
1.2

200 240 280 320 360 400 440 480

WAVELENGTH / nm

ABSORBANCE (simulated)

200 240 280 320 360 400 440 480

WAVELENGTH / nm
Fig. 1. (a) The linear connection of the 13 sampling points (‘wavelengths’) for the simulated training spectra from Table 1
LEARNl-4 (a) and LEARNS-8 (+I shows two groups of spectra with a certain within-class variation. (b) A visual comparison
with Fig. 2a illustrates the graphical class membership of the test spectrum TEST1 ( - ) to group LEARNl-4 and of TEST2 (+ ) to
group LEARN%% TEST3 (0) seems not to belong to any of the hvo groups of the training spectra.
318 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329

weak separation. For the other remaining optical complete set of 24 spectra was presented 30 times
regions no univariate separation can be observed in random order to the network. A maximum of
in Table 4. 16 epochs was found enough to reach full training
convergence for the slowest learning rate n = 0.1.
4.2.3. Vigilance parameter p and learning rate q This observed fast training convergence of the
As already shown in the simulation study (Ta- neural networks weight matrices is in accordance
ble 3) an increasing value of pmax provides a finer with the general known fact that ART based
resolution of the data set into more subclusters. neural networks can be trained much faster than,
The learning rate 77 as linear parameter in the for example, backpropagation rule based MLF
learning rules (1) and (2) is expected to have an neural networks. A more impressive example for
influence on the category formation, too. To an- fast convergence of ART neural networks is dis-
swer this question more quantitatively 77has been cussed by Carpenter and Grossberg in ref. [42]
systematically varied for the classification prob- where a speed ratio of 5
lem of the 24 binary coded UV-Vis spectra from
Table 4. To do this the neural network was de-
signed with 13 input units corresponding to the
13 wavenumbers and a large enough chosen pool
of 100 output units. However, this pool presents
only potential output units and the network will
decide by itself whether it needs output units
from this pool or not. For every setting combina-
tion of n and p the network was trained. The

0.3
l 20
l 14
l 17
T 0.2 - * * * 18
.m
c 13 * 19 24 * 16
2 * 21 * 22
Eli 0.1 - * 15
x l23
d

ii O
z 07 06
g -0.1 - OS 08
g 0 10

2 -a2 - o3 04 09

si
g -0.3 - 01 02

0 12
0 11
-0.4 I
0.05 0.1 0.15 0.2 0.25 0.3 0.35
PRINCIPAL COMPONENT 1 (47.2 % variance)

Fig. 2. Principal components score plot of the 24 experimental UV-Vis spectra from Table 3 in the space of the first hvo principal
components. The numbers correspond to the mixture spectra MIXl-12 (0) and to the sample numbers of the mixture spectra
LINC013-24 (* ).
 D. Wienke, G. Kateman /Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329 319

to ‘neglect’ a large part of its previously learned

number of formed clusters
26 knowledge. For n > 1.25, it tends to forget the
1 20
most recently trained knowledge, too. This large
20- n resulted for the 24 UV-Vis spectra, for exam-
ple, in a generation of continuously new class
16 - boxes. After one epoch for n = 1.25 24 output
neurons were active. After a second epoch 48
10 - output neurons, etc. From the finally selected
classification results (Table 5) it can be seen how
larger clusters tend to split into more smaller
subclusters if n is forced. In Table 5 (column
n = 0.5) the MIX group splits from five clusters
6.1 0.3 0.6 0.7 0.9
(2,4,5,6,8) into seven clusters (2 >5 3677 78710,13) for
learning rate v = 0.7. One can further see that the MIX and
z the LINCO groups do not share any common
Fig. 3. Effect of increasing learning rate n on the number of
cluster. In other words, the formation of new
detected clusters in the binary coded experimental UV-Vis
spectra from Table 4 using all 13 wavenumbers as input subgroups occurs separately for each class. This
features to the ART-l artificial neural network. means that no ‘class box overlap’ between the

Table 4
In the pattern recognition study used binary coded experimental UV-Vis spectra from Table 3 (For details of the coding procedure
see text)
Sample Wavenumber/cm-’
19.0 19.5 20.0 20.5 21.0 21.5 22.0 22.5 23.0 23.5 24.0 24.5 25.0
MIX1 1 0 0 0 0 0 1 0 1 1 1 1 0
MIX2 1 0 0 0 0 0 1 1 1 1 1 1 0
MIX3 0 0 0 0 0 0 0 0 0 0 0 1 0
MIX4 1 0 0 0 0 0 0 0 0 0 1 1 0
MIX5 1 1 0 0 0 0 0 0 0 0 1 1 0
MIX6 1 1 1 0 0 1 1 1 0 1 1 1 0
MIX7 0 0 0 1 1 0 1 0 0 1 1 1 0
MIX8 0 1 1 1 1 1 1 0 1 1 1 1 0
MIX9 0 0 0 1 0 0 1 0 1 1 1 1 0
MIX10 1 1 1 1 1 1 1 1 1 1 1 1 0
MIX1 1 0 0 0 0 0 0 0 0 1 1 1 1 0
MIX12 0 0 0 0 0 0 0 1 1 1 1 1 0
LINC013 0 0 0 0 0 1 1 0 0 0 0 0 1
LINC014 0 0 1 1 1 ’ 1 0 1 0 0 0 0 1
LINC015 0 1 1 1 0 0 1 0 0 0 0 0 1
LINC016 1 1 1 1 1 1 1 1 1 1 1 0 1
LINC017 0 1 1 1 1 1 0 1 1 0 0 0 1
LINCOl8 1 1 1 1 1 1 1 1 1 1 0 0 1
LINC019 0 0 0 1 1 0 0 1 0 0 0 0 1
LINC020 1 1 1 1 1 1 1 1 1 1 0 0 1
LINC021 1 1 0 0 1 1 0 1 0 0 0 0 1
LINC022 1 1 1 1 1 1 1 1 1 0 0 0 1
LINC023 0 0 1 0 0 0 0 0 1 0 0 0 1
LINC024 1 1 1 0 1 1 1 0 1 0 0 0 1
320 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329

MIX and the LINCO groups exists. This result
agrees with the graphical impression from the
PCA scores plot (Fig. 21.
4.2.4. Interpretation of the weight matrices
The similarity between all spectra that were
classified in the same ith class box with its corre-
sponding ith column in Wl and with its ith row
in W2 becomes obvious by a look into Hinton
diagrams (Fig. 4). The largest weights announce
the most frequently appearing joint features for
one class i. Small weights indicate unimportant
features for that class. From Fig. 4 it is obvious
that the columns 5-24 within Wl and the rows
5-24 within W2 have the same constant size.
That simply means that only the k = 4 output
neurons are linked to the network. All other
neurons (k > 4) belong to the pool of potential
but actually not required output neurons. When a
special pattern recognition task has to be solved
in the future, for example, that requires k > 4
output neurons then the network will be able to
link additional output units from that pool to its
network structure.
4.2.5. Effect of the selection of input features on
the ART-l neural network’s self-organizing behau-
ior
The classical univariate Fisher weights [53] as
given in Fig. 5 as one kind of expression for the
separability of the MIX group from the LINCO
show the optical range > 23 500 cm-’ to be the
most differentiating one. This corresponds to the
graphical impression of univariate sorting of ze-
ros and ones in the binary coded data set (Table
4) at the position of these wavenumbers. The
optical range from 22000 cm-’ up to 23 500
cm-’ (Fig. 5) shows in contrast to that low Fisher
weights as expression for a kind of random order
of the MIXl-12 and LINC013-24 spectra.
The following results of a wavenumber selec-
tion experiment (Table 6) with the 24 UV-Vis
spectra illustrate how the rejection of subsets of
wavenumbers influences the self-organizing be-
havior of an ART based neural network. A step-
wise elimination of wavenumbers with a low in-
formation content about the separability of the
MIX and the LINCO classes from column 1
(Table 6) to column 5 (Table 6) provides fewer

.7 qBe3mEEiEEiBm~
t-4q S q 2gs q ~BBEBB
m q BmBmEEIEaBBBBB'
a q mmmmmmmmm~ q
q mmmmmmmmmmmm
q mmmmmmmmmmmm
q mmmmmmmmmmmm
q mmmmmmmmmmmm
.- rlq mmmmmmmmmmmm
q mmmmmmmmmmmm
q mmmmmmmmmmmm

L__
m q mmmmmmmmmmmm
u-l q mmmmmmmmmmmm
6 q mmmmmmmmmmmm
q mmm03mmmmmmmm
q mmmmmmmmmmmm
d q mmm~mmmmmmmm
q mmmammmmmmmm
q mmmr3mmmmmmmm
q mmmmmmmmmmmm
q mmmmmmmmmmmm
q mmmmmmmmmmmm
q BmBBBBBBmBBm
q mmmmmmmmmmmm
19000 -22Mx) 2X00
CLASS i @ WAVENUMBER j/cm’
Fig. 4. Graphical presentation of the weight matrices Wl and WZ of a trained ART-l neural network by Hinton diagrams. Every
single ‘cross box’ corresponds to one weight respective to one matrixelement in Wl (a) or WZ (b). The surface of every single cross
box corresponds to a number between 0 (no box) and 1 (large box). The network was trained with the 24 binary coded experimental
UV-Vis spectra from Table 4 using 13 wavenumbers as input features (vigilance parameter p = 0.5, learning rate q = 0.1).
 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329 321

Table 5 4.3. Experimental 2. Infrared reflectance spec-

Effect of increasing learning rate 11 on the individual class troscopy of ternary A120,-CaO-SO, glasses
membership of all 24 experimental UV-Vis spectra (Tables 3
and 4) obtained by the ART-l neural network for constant
vigilance parameter p = 0.5 and by using all 13 wavenumbers 4.3.1. Results of a multivariate statistical study
as input features (see also Fig. 4). The numbers represent the A recent study of the relations between the 66
name of a cluster given by the network and in this way the glass spectra performed by Wienke 1471 showed
individual class memberships for a sample in this cluster that the 69-dimensional wavenumber space can
Sample 17 be mapped down to a two- or three-dimensional
0.1 0.3 0.5 0.7 principal component space (Fig. 6a). By means of
MIX1 2 5 5 6 a hierarchical cluster analysis using Ward’s
MIX2 2 5 5 6 method [57-591 two well separated large clusters
MIX3 2 2 2 2 including 39 and 27 glasses were detected in the
MIX4 2 2 2 2
direction of the second principal component (Fig.
MIX5 2 2 2 2
MIX6 3 4 4 5
6b). A correlation analysis of all spectral principal
MIX7 2 7 6 8 components to the gross composition of the
MIX8 3 7 8 10 glasses yielded the result that the second princi-
MIX9 2 5 5 8 pal component is highly correlated (R = 0.944)
MIX10 3 8 8 13
with the Al,O, concentration. Moreover, the cor-
MIX11 2 5 5 7
MIX12 2 5 5 7 responding VARIMAX rotated eigenvector
(‘varivector’ 1531) of principal component 2
LINCOl 1 3 7 4
showed - except in the region between 590-610
LINC02 1 1 1 3
LINC03 1 3 3 1 cm-’ - a spectral band which exactly corre-
LINC04 4 6 10 15 sponds to the maximum position of the main
LINC05 1 1 1 12 band in the reflectance spectrum of pure (Y-
LINC06 4 6 9 14 Al,O,. These results obtained from the classical
LINC07 1 1 1 3
multivariate statistical analysis of the 66 FT-IRR
LINCOS 4 6 9 14
LINC09 1 1 7 9 spectra were in agreement with the hypothesis of
LINCOlO 4 6 9 14 Yamane and Okuyoma [54] about an abrupt in-
LINCOll 1 3 3 1
LINC012 1 3 7 11
Fisher weight
31

2.5
but larger classes. This gives a purer separation 2.27
2.18
into fewer class boxes. A stepwise elimination of 2
wavenumbers with high separation power (from
column 6 to 9, Table 6) provides in contrast to
that an increasing number of subclusters up to a
sharing of common clusters between both groups
of spectra. The pattern MIX2, MIXlO, LINC04,
LINC06 and LINC08 were now wrongly classi-
fied, for example, in one common class box 7
19 19.5 20 20.5 21 21.5 22 22.5 23 23.5 24 24.5 25
(Table 6, column 8). The results show how an
ART-l neural network can fail in its self-organiz-
ing behavior if the set of input variables (here: a wavenumber / lOOO*cm
-1

suitable set of selected wavenumbers) has been

Fig. 5. Univariate Fisher weights (height of the bars) as
chosen without any strategy. Not all of the wrong statistical measure for the separation between the MIX and
or noisy information can be compensated for by the LINCO spectra calculated for all 13 wavenumbers (data
simple adaptation of some corresponding weights. taken from Table 3).
322 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329

crease of the coordination of A14+ to A16+ by
increase of the relative Al,O, concentration in
the glasses to values higher than lo-15 m%. A
higher coordination number of A16+ causes a
changed environment for the Al atom. This can
be observed as discontinuities in the X-ray Al K,
fluorescence shifts f-541.The new molecular bonds
of the Al species to its neighbors cause tensions.
This causes finally a change of the glass network
structure, too. This abrupt change can be ob-
served as smah simuItaneous changes in the ab-
sorbances over the entire FT-IRR spectrum pro-
viding a clustering of the 66 spectra (Fig. 6a and
b).
4.3.2. Results of an ART-l study
The 66 FT-IRR spectra were binary coded as
described above for the UV-Vis spectra. Then
they were presented to an ART-l neural network
in random order having input units corresponding

Table 6
Effect of directed selection of subsets of wavenumbers as input features on the individual class membership of all single 24
experimental W-Vis spectra (Tables 3 and 4) obtained with the ART-l artificial neural network (constant p = 0.25, constant
1) = 0.3)
Name of the Used subset of wavenumbers/cm-’
spectrum 22000 22000 22000 22000 22000
22500 22500 22500 22500 22500 22500
23000 23000 23000 23000 23000 23000 23000
23500 23500 23500 23500 23500 23500 23500 23500
24000 24000 24000 24000 24000 24000
24500 24500 24500 24500 24500 24500 24500
25000 25000 25000 25000 25000 25000 25000
MIX1 2 2 2 2 2 2 2 2 5
MIX2 4 2 2 2 2 4 4 7 9
MIX3 2 2 2 2 2 2 2
MIX4 2 2 2 2 2 2 _ 2
MIX5 2 2 2 2 2 2 _ 2
MIX6 4 2 2 2 2 4 4 6 4
MIX7 2 2 2 2 2 2 2 2 2
MIX8 4 2 2 2 2 4 2 2 5
MIX9 4 2 2 2 2 4 2 2 5
MIX10 4 2 2 2 2 4 4 7 9
MIX11 2 2 2 2 2 2 2 2 5
MIX12 4 2 2 2 2 4 4 5 6

LINCOl 1 1 1 1 1 1 1 1 1
LINC02 1 1 1 1 1 1 3 3 3
LINC03 1 1 1 1 1 1 1 1 1
LINC04 3 4 4 3 3 5 6 7 8
LINCOS 3 3 3 1 1 3 3 4 3
LINC06 3 3 3 3 1 3 6 7 7
LINC07 1 1 1 1 1 1 3 3 3
LINCO8 3 3 3 3 1 3 6 7 7
LINC09 1 1 1 1 1 1 3 3 3
LINCOlO 3 3 3 1 1 3 5 4 7
LINCOll 1 1 1 1 1 1 1
LINCOl2 1 1 1 1 1 1 1 1 1
LINC012 1 1 1 1 1 1 1 1 1
The block on the left-hand side (columns l-5) illustrates the influence of the stepwise rejection of wavenumbers with low
separation power (see Fig. 7). The block on the right-hand side (columns 6-9) shows the effect of the elimination of wavenumbers
with high separation power. (-: corresponding spectrum cannot be classified because the value of all used input features was equal
to zero, see Table 4, text and Eqs. (3) and (41.1
 D. Wienke, G. Kateman /Chemometrics and intelligent Laboratory Systems 23 (1994) 309-329 323

to the m = 69 chosen wavenumbers. By using a the weights and a formation of five clusters was
learning rate q = 0.1 and a vigilance parameter obtained (Fig. 64. To be sure about complete
of p = 0.5 after 18 epochs a stable convergence of convergence an upper limit of at least 30 training

@lo3 34

46
-$2 0.2 ;0
‘C
9
3Y l9
22 4 *g
11 38 41
; 0.1 31 ’ 17
P)
77
263 534510 3
272g 1631058
2 15
g O
33 00 Qj

B
42 32~21351p
61
8
a -0.1

44 60 362&J7 4o 24 25
a

2 49 56 35 sfg 2150 237

g -0.2 62
37

-0.3
0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13 0.14 0.15 0.16 ;7
PRINCIPAL COMPONENT 1[ 54.14% variance)
a

g3 0

7 0.2 0 0

‘L 8’ o
$
0 o 0
0 0 cl
0.1 0 OO 0
0
0 00 0
08 00
0 oo”
0
ig 0 e 0 0

& 00 ,”
X
8 x OO
U
2 -0.1 X X X
X X

xI% xx

X XX XXX
2 X x

z -0.2 X

I
-0.3
0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17
PRINCIPAL COMPONENT 1(54.14% variance]
Fig. 6. Map of the 66 experimentat FT-IR reflectance spectra onto the space of the first two principal components extracted from
the 69-dimensional space of autoscaled features. (aI Numbers correspond to the sampie numbers; (b) index o and x corresponds to
two large sample groups found by Ward’s hierarchical clustering method; (c) index l-5 corresponds to the class membership as
found by the ART-l artificial neural network.
 324 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329

-0.3
0.06 0.07 0.08 0.09 0.1 0.11 0.12 0.13 0.14 0.15 0.16 0.17
PRINCIPAL COMPONENT 1 ( 54.14 % variance)
Fig. 6. (continued).

Table 7
Most important (in the IR region of 400-1200 cm-’ observable) vibrations of a ternary AlaO,-CaO-SiO, glass system taken from
refs. [49,50]
Element Wavenumber range/cm-’ Observed infrared vibration
Si 400-600 Si-0-Si, bond and rocking motions
Si-O-, nonbridged and at the end position
500 SiO,, 6 motion
790 Si-0-Si bond motion
800-1200 Si-0-Si bond motion
Si-O- bond motion
800,1100 Non-symmetrical, symmetrical bond motion
900-930 Isolated SiO, tetraeders
1050 Si-0-Si

1030 Si-O-
1050 Si-O...H
1090-1120 Si-0-Si
10.52-1111 Si-0-Si
909-970.. 1000 Si-O-

Al 1050 Si-0-Si
800 Characteristic broadening of the non-symmetrical Si-0-Si bond motion
700-740 Formation of an additional band (difficult to identity)
830 Small band if SiO, is present
1000 Intensive band if SiO, is present
620-1250 Characteristic broadening of already existing bands
980 Al-0

Ca 1050 Si-0-Si
800 Coupling of SiO- with non-symmetrical motion (enhanced total intensity)
 D. Wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329 325

I?
3
I

OR 0.8

._
‘2 Is
; “.6 r 0.6

0.4

0.2

0
1200 1130 1061 991 922 852 763 713 643 514 504 435
e- WA”EN”MBER /cm-’

1.2

0.8 0.8

.-

gi6

cl4 0.4

0.2

O-
1200 ,130 1061 991 922 a52 la3 713 643 574 504 435 1200 1130 1061 991 922 852 783 713 643 574 504 435
a WAVENUMBERlcrd c- WAVENUMBER/cd

0.8

2
; 0.0

0.4

0.2

0
12iN 1130 1061 991 922 852 703 713 643 574 504 435
- WAVENUMBER /cm-’

Fig. 7. Bargraph presentation of the weight vectors Wlj,,, corresponding to the i = 1 . .5 output neurons and j = 1.. .69 input
neurons. The weights were obtained after the training with the 66 experimental fl-IRR spectra and unsupervised formation of five
classes by an ART-l artificial neural network.
326 D. Wienke, G. Kateman /Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329
epochs was chosen. The training run was re-
peated 10 times and in every case the same 5
clusters with the same individual class member-
ship of all pattern vectors were obtained. Com-
paring Fig. 6c (index 1, 2) with Fig. 6b (index o, x>
two similar large and located clusters along the
second principal component axis are observed.
This corresponds well to the results of the classi-
cal multivariate analysis. Only in region between
the two clusters the ART-l neural network gen-
erates three subgroups (index 3,4,5) according to
the chosen size of 77 and p. This hints towards
probably ‘mixed’ substructures in the glasses with
an A14” as well as an A16+ coordination. A look
at the weights provides more quantitative infor-
mation about the main spectral properties of the
five single clusters (Fig. 7a-d). First, it is obvious
that the main important weights belonging to
output neuron 1 (Fig. 7a, and Fig. 6c class index
1) can be related to the optical range between
580 and 1050 cm-‘. This corresponds to the
position of the main optical band of a-Al,O,
(Table 7) and to the results as found in the
multivariate study. Class 2 (output neuron 2) in
Fig. 6c corresponding to low bruto contents of
AI,O, shows the most important weights for the
complementary optical regions of 435-580 cm-’
and 1080-1200 cm-* (Fig. 7b). In these regions
the main vibrations of Si within the glass are
found (Table 7). It is well known, too, that pure
SO, shows its two main bands in these two
optical regions and a very small band between
600 and 800 cm- ‘. A look at the weights of
output neuron 3 (Fig. 7c, class 3) shows four main
optical regions: 435-570 cm-‘, 600-700 cm-‘,
800-1080 cm-’ and 1100-1200 cm-‘. This corre-
sponds with the four bands as they can be found
in the IT-IRR spectrum of pure CaO as recently
discussed by Wienke 1471.The other two classes 4
and 5 (output neurons 4 and 5) show ‘mixed’
structures and will not be discussed here.
5. Conclusions
Grossberg’s adaptive resonance theory based
artificial neural network (ART-11 has been eluci-
dated for its potential use in chemical pattern
recognition.
In contrast to all other types of artificial neural
networks an ART neural network has a less pre-
defined and less rigid structure. However, during
the training the network adapts not only its
weights but also its entire structure to the offered
data. The output of the training are classes of
training pattern and the corresponding class
memberships for every training pattern. This is
similar to a result that would be obtained from
any classical unsupervised working pattern recog-
nition technique [57-591. However, as shown in
the present study, the direct interpretable weight
vectors of a trained ART network carry sup-
pressed information about characteristic proper-
ties of the classes. Based on these weights an
ART network can be used like a supervised work-
ing classifier, too. During the prediction step the
network continuously keeps learning (plasticity
and adaptability) but it is able to form on-line
additional class boxes if required keeping saved
the previously trained information (stability and
robustness). A simulation study and a processing
of experimental UV-Vis and FT-IR spectra by
ART demonstrated that a suitable selection of
variables for the training vectors forces the for-
mation of less clusters with a purer separation.
The same effect has been achieved by a choice of
low values for vigilance parameter p and learning
rate q.
Compared to other pattern recognition meth-
ods the use of ART based neural networks
promises the following advantages: (i) The unsu-
pervised nature of the training behavior saves
development time to find a suitable structure for
a desired supervised classifier. (ii) The process of
competitive learning with the step of compari~n
between xi and x,? looks like the kth nearest
neighbor clustering method (KNN) 1551. How-
ever, the (n * k)/2 comparisons of the y1 input
patterns with k weight vectors of the classes with
k c n in place of [(n * n) - nl/2 pair-wise com-
parisons of all input patterns provides a signifi-
cant gain in computational speed for ART com-
pared to KNN. The observed high training speed
of less than 20 epochs for the used data sets
increases the gain in time and additionally
 D. wienke, G. Kateman / Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329
promises a potential on-line applicabili~ in run-
ning chemical processes. (iii) One training vector
only is enough to initialize the formation of a
class box. (iv) The concept of adaptive resonance
makes an overtraining theoretically impossible.
This is in agreement with the observations in the
present work, too. (v) Significantly deviating
training patterns (outliers) cause the formation of
additional class boxes. In this way the generation
of tensions or empty regions in the trained class
boxes are avoided.
On the other hand, the wavelength selection
experiment demonstrated some limitations of
ART: (i) The cluster structure of the input data
forces the structure of the network. This means
that noise can only partly be compensated for by
a simple adaptation of weights. Very noisy data
or a lot of outliers cause the formation of numer-
ous subclusters. (ii) The development of a con-
cept of individual vigilance parameters p for the
description of the individual sizes of different
class boxes could be useful. At this moment, for
example, ART is modeling curved hyperboxes by
a series of regular shaped hyperboxes.
However, this has been a first study about the
principles of ART for chemical pattern recogni-
tion demonstrated by processing UV-Vis and
FT-IR spectra. In the upcoming research the
ART-2 based artificial neural network will be
studied 1601.
Acknowledgement
The authors are grateful to G. Pflug and H.
Dunken (University of Jena, Germany) for offer-
ing the FT-IR reflectance spectra to the authors,
The authors would like to thank both referees
for their valuable hints and comments.
References
[l] G. Kateman, Evolutions in Chemometrics, Analyst, 115
(1990) 487-493.
[2] D. Wienke, C. Lucasius, M. Ehrlich and G. Kateman,
Multicriteria optimization of analytical procedures using
a genetic algorithm, Analytica Chimica Acta, 271 (1993)
253-268.
327
[3] PC. Jurs, B.R. Kowalski and T.L. Isenhour, Computer-
ized learning machine applied to chemical problems-
Molecular formula determination from low resolution
mass spectrometry, Analytical Chemistry, 4lf1969) 21-26.
[4] T.J. Stonham, I. Aleksander, M. Camp, W.T. Pike and
M.A. Shaw, Classification of mass spectra using adaptive
digital learning networks, Analytical Chemistry, 47 (1975)
1817-1822.
[5] J.U. Thomson and B. Meyer, Pattern recognition of the
1H NMR spectra of sugar alditols using a neural net-
work, Journal of Magnetic Resonance, 84 (1989) 212-217.
[6] L.S. Anker and P.C. Jurs, Prediction of Cl3 NMR chemi-
cal shifts by artificial neural networks, Analytica! Chem-
&try, 64 (1992) 1157-1164.
[7] M. Bos, A. Bos and W.E. v.d.Linden, Processing of
ion-selective electrode array signals by a neural network,
Analytical Chimica Acta, 233 (1990) 31-39.
[8] T. Nakamoto, K. Fukunishi and T. Moriizumi, Identifica-
tion capability of odor sensor using quartz resonator
array and neural network pattern recognition, Sensors
and Actuators, Bl (1990) 473-476.
[9] SM. Chay, Y. Iwasaki, M. Suzuki, E. Tamiya, I. Karube
and H. Maramatsu, Detection of odorants using an array
of piezoelectric crystals and neural network pattern
recognition, Analytical Chimicu Acta, 249 (1991) 323-329.
[lo] M. Bos and H.T. Weber, Comparison of the training of
neural networks for quantitative X-ray fluorescence spec-
trometry by a genetic algorithm and backward error
propagation, Analytical Chimica Acta, 247 (1991) 97-105.
[II] M. Glick and G.M. Hieftje, Classification of alloys with
an artificial neural network and multivariate calibration
of glow discharge emission spectra, Applied Spectroscopy,
45 (1991) 1706-1716.
[12] M. Otto, T. George, C. Schierle and W. Wegscheider,
Fuzzy logic and neural networks - Applications to ana-
lytical chemistry, Pure and Applied Chem~t~, 64 (1992)
497-502.
[13] C. Schierle, M. Otto and W. Wegscheider, A neural
network approach to qualitative analysis in inductively
coupled plasma atomic emission spectroscopy (ICP-AES),
Fresenius’ Journal of Analytical Chemistry, 343 (1992)
561-565.
[14] E.W. Robb and E.M. Munk, A neural network approach
to infrared spectrum interpretation, Microchimica Acta
Wien), I(19901 131-139.
[15] B.J. Wythoff, S.P. Levine and S.A. Tomellini, Spectra
peak verification and recognition using a multilayered
neural network, Ana~ticai Chern~t~, 62 (1990) 2702-
2709.
(161 J.R.M. Smits, P. Schoenmakers, A. Stehmann, F. Sys-
termans and G. Kateman, Interpretation of infrared
spectra with modular neural network systems, Chemo-
metrics and Intelligent Laboratory Systems, 18 (1993) 27-
39.
[17] M. Meyer and T. Weigelt, Interpretation of infrared
spectra by artificial neural networks, Analytica Chimica
Acta, 265 (1992) 183-190.
328 D. Wienke, G. Kateman /Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329
[18] K. Tanabe, T. Tamura and H. Uesaka, Neural network
system for identification of infrared spectra, Applied
Spectroscopy, 46 (1992) 807-810.
1191 R.J. Long, G.V. Gregoriou and J.P. Gemperline, Spec-
troscopic calibration and quantitation using artificial neu-
ral networks, Analytical Chem~t~, 62 (1990) 1791-1797.
[20] P.J. Gemperline, J.R. Long and V.G. Gregoriou, Nonlin-
ear multivariate calibration using principal components
regression and artificial neural networks, Analytical
Chem~t~, 63 (1991) 2313-2323.
[21] J.R. Lang, H.T. Mayfield, M.V. Henly and P.R. Kro-
mann, Pattern recognition of jet fuel chromatographic
data by artificial neural network with backpropagation of
errors, Analyttcal Chemistry, 63 (1991) 1256-1261.
[22] H. Lohninger, Classification of mass spectral data using
neural networks, in J. Gmehling (Editor), Software Deuel-
opment in Chemistry, Vol. 5, Springer, Berlin, 1991.
1231J. Zupan and J. Gasteiger, Neural networks: A new
method for solving chemical problems or just a passing
phase?, Analytica, Chimica Acta, 248 (1991) l-30.
[24] P.A. Jansson, Neural networks: An overview, Analytical
Chem~t~, 63 (1991) 357A-362A.
1251 M. Gross and F. Seibert, Neural network for image
analysis of environmental protection, in R. Denzer (Edi-
tor), tisualisierung van Umweltdaten, Springer, Berlin,
1991.
[26] P. Arrigo, F. Giuliano, F. Scaha, A. Rapallo and G.
Damiani, Identification of a new motif on nucleic acid
sequence data using Kohonen’s self-organizing map,
Computer Applications in the Biosciences, 7 (1991) 353-
357.
[27] V.S. Ross, I.F. Croall and H.J.H. Maefic, An application
of unsupervised neural network methodology (Kohonen
topology preserving mapping) to QSAR analysis, Quanti-
tative Structure- Activity Relationships, 10 (1991) 6-1.5.
[28] W.J. Melssen, J.R.M. Smits, G.H. Rolf and G. Kateman,
Two-dimensional mapping of IR spectra using a parallel
implemented Kohonen network, Chemometrics and Intel-
ligent Laboratory Systems, 18 (1993) 195-204.
[29] P. de B. Harrington, Minimal neural networks; Differen-
tiation of classification entropy, Chemometrics and Intelli-
gent Laboratory Systems, 19 (1993) 143-154.
1303C. Borggaard and H.H. Thodberg, Optimal minimal neu-
ral interpretation of spectra, Analytical Chemistry, 64
(1992) 545-551.
(311 S. Grossberg, Studies of Mind and Brain, Reidel, Dor-
drecht, 1982.
[32] S. Grossberg, The Adaptive Brain I - Cognition, Learn-
ing, Reinforcement and Rhythm (Advances in Psychology
Series, Vol. 421, North Hoiiand, Amsterdam, 1987.
[33] S. Grossberg, The Adaptiue Brain II - Cognition, Leam-
ing, Reinforcement and Rhythm (Advances in Psychology
Series, Vol. 431, North Holland, Amsterdam, 1987.
[34] S. Grossberg and M. Kuperstein, Neural dynamics of
adaptive sensor-motor control, in Neural Ne~orks Re-
search and Applications, Pergamon Press, New York,
1989.
[35] S. Grossberg, Adaptive pattern classification and univer-
sal recoding, I. Parallel development and coding of neu-
ral feature detectors, Biological Cybernetics, 23 (1976)
121-134.
[36] G.A. Carpenter and S. Grossberg, ART2, self-organiza-
tion of stable category codes for analog input patterns,
Applied Optics, 12 (1987) 4919-4930.
[37] G.A. Carpenter and S. Grossberg, A massively parallel
architecture for a self-organizing neural pattern recogni-
tion machine, Comp~fer vision, Graphics and Image Pro-
cessing, 37 (1987) 54-115.
[38] G.A. Carpenter and S. Grossberg, The ART of adaptive
pattern recognition by a self-organizing neural network,
IEEE Computer, 21(3) (1988) 77-78.
[39] G.A. Carpenter and S. Grossberg, ART-3, hierarchical
search using chemical transmitters in self-organizing pat-
tern recognition architectures, Neural Networks, 3(2)
(1990) 129-152.
(401 S. Grossberg, Attention and recognition learning by
adaptive resonance, Behavioral and Brain Sciences, 13(2)
(1990) 241-241.
1411G.A. Carpenter, S. Grossberg and D.B. Rosen, ART-2A,
an adaptive resonance algorithm for rapid category Iearn-
ing and recognition, Neural Nefworks, 4 (1991) 493-504.
1421 G.A. Carpenter, S. Grossberg and D.B. Rosen, FUZZY-
ART, fast stable learning and categorization of analog
patterns by an adaptive resonance system, Neural Net-
works, 4 (1991) 759-771.
[43] L.I. Burke, Clustering characterization of adaptive reso-
nance, Neural Networks, 4 (1991) 48.5-491.
[44] K.W. Gan and K.T. Lua, Chinese character classification
using an adaptive resonance network, Pattern Recogni-
tion, 25 (1992) 877-882.
(451 G.A. Carpenter, S. Grossberg and J.H. Reynolds, ART-
MAP: Supervised real-time learning and classification of
non-stationary data by a self-organizing neural network,
Neural Networks, 4 (1991) 565-588.
1461 H.M. Koster, Die chemische Silikatanalyse - Spek-
tralphotometrische, komplexometrische und ftammenfo-
tometrische Analysen-Methoden, Springer, Berlin, 1979.
[47] D. Wienke, Application and Development of Multivariate
Statistical Methods for Trace Analysis; Structure Analysis
and Glass Technology (Ph.D. Thesis), University of Jena,
Jena, 1990.
[48 R. Stephanowitz, Theoretische und Experimentelle Unter-
suchungen zur Anwendung der IR Re~~~~spektroskopie
zur ~jektiven Charukte~sierung van Giasobe~~chen
(Ph.D. Thesis), University of Jena, Jena, 1987.
[49] H. Dunken, Physikalische Chemie der Glasoberjlache,
Deutscher Verlag fiir Grundstoffindustrie, Leipzig, 1981.
[SO] D.M. Sanders, W.B. Person and L.L. Hench, Quantita-
tive analysis of glass structure with the use of infrared
reflectance spectroscopy, Applied Spectroscopy, 28 (1974)
247-259.
[51] C. Moier, S. Bangert, S. Kleiman and J. Little, MATLAB
User Guide, The MathWorks Inc., Natick, MA, 1990.
[52] 11. Demuth and M. Beale, Neural Network TOOLBOX
 D. Wienke, G. Kateman /Chemometrics and Intelligent Laboratory Systems 23 (1994) 309-329 329

For Use with MATLAB, User Guide, The MathWorks Einfiihrung in Methoden und Verfahren der automatischen
Inc., Natick, MA, 1992. IUassifikation, De Gruyter, Berlin, 1977.
1531 M.A. Sharaf, D.L. Illrnan and B.R. Kowalski, Chemomet- [58] H. Splth, Clusteranalyse Algorithmen zur Objektklassi-
rics, Wiley, New York, 1986. fizierung und Datenreduktion, Oldenburg, Miinchen, 1977.
1541 M. Yamane and 0. Okuyoma, Coordination number of [59] M. Goldstein and W.R. Dillon, M&variate Analysis,
aluminium ions in alkali-free aluminosilicate glasses, Methods and Applications, Wiley, New York, 1984.
Journal of Non-Crystalline Solids, 52 (1982) 217-223. [60] D. Wienke, Y. Xie and P.K. Hopke, An adaptive reso-
[Xl K. Varmuza, Pattern Recognition in Chemistry - Lec- nance theory based artificial neural network (ART-2A)
ture Notes in Chemistry, Springer, Berlin, 1978. for rapid classification of airborne particles by their SEM
1561 P.J. Rousseuw and M.A. Leroy, Robust Regression and images, Chemometrics and Intelligent Laboratory Systems,
Outlier Detection, Wiley, New York, 1987. submitted for publication..
1571 D. Steinhausen and K. Langer, Clusteranalyse -