See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/341735137

Associative-projective neural networks: architecture, implementation,

applications

Conference Paper · November 1991

CITATIONS READS
19 73

3 authors, including:

Ernst Kussul Dmitri A. Rachkovskij

Universidad Nacional Autónoma de México National Academy of Sciences of Ukraine
3 PUBLICATIONS   23 CITATIONS    70 PUBLICATIONS   1,057 CITATIONS   

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Dmitri A. Rachkovskij on 29 May 2020.

The user has requested enhancement of the downloaded file.


ASSOCIATIVE-PROJECTIVE NEURAL NETWORKS :
ARCHITECTURE, IMPLEMENTATION, APPLICATIONS
Ernst M. Kussul, Dmitri A. Rachkovskij & Tatyana N. Baidyk
Ukrainian Academy of Sciences / Institute of Cybernetics - Kiev, USSR
Abstract : In this paper we consider an original neural network paradigm based on an
associative-projective neural network architecture. The architecture provides a uniform
basis for intelligent systems working in various fields. We describe several modes of
architecture operation. This architecture is supported by high performance neurocomputers.
Experiments carried out to test the architecture characteristics and its efficiency in applica-
tions are reviewed. We also discuss application examples of this architecture for image
texture classification and an approach to visual object recognition.
Keywords : neural network architecture, artificial intelligence problems, neurocomputers,
neural network applications.
463
1. INTRODUCTION
Well-known up-to-date neural network architectures may be subdivi-
ded into several basic categories by their purpose, e.g. neural
networks for neurobiological simulations, for information prepro-
cessing, for pattern recognition; networks for concept formation,
situation estimation, etc. Some problems of Artificial Intel-
ligence still remain uncovered by the neural network solutions,
including action planning, logical inference, formation of complex
hierarchical structures for knowledge bases, natural language
understanding and others. So far these problems demand to use
algorithms of traditional AI.
Our goal is to develop neural network architecture that is
equally destined for solution of all basic AI problems. Such
architecture permits to realize a common approach to heteroge-
neous information processing at different hierarchical levels,
allows to concentrate on elaboration of specialized high
performance neurocomputer for its support, and provides a uniform
basis for novel intelligent systems design.
The basic design principles for the proposed architecture and
its application examples will be described below.
2. ASSOCIATIVE-PROJECTIVE NEURAL NETWORK ARCHITECTURE
We build neural network from separate blocks called neural fields.
There are three types of neural fields: associative field (AF),
buffer field (BF) and difference field (DF). Each neural field
consists of n binary neurons. The fields are linked by bundles of
n non-modifiable projective connections. Each projective connec-
tion has a unity weight and connects the output of a neuron from
one field and the input of the neuron with the same number Егоз
another field.
Neurons of buffer and difference fields actually are clocked
RS triggers. The priority of the reset input is higher than that
of the set input. The whole bundle of connections may be coupled
to either reset or set inputs of the field neurons. Hence we'll
distinguish the bundles of reset and set connections.
Projective connections may be in two states: "on" or "off".
In "on" state field activity patterns are transferred through
projective connections and in "off" state projective connection»
are blocked. Special algorithms control switching of projective
connections for the present. In the future we suppose to develop
special neural structure for this purpose.
Buffer neural field is able to receive and store patterns of
neuron activity from the other fields and executes some other
auxiliary functions. Difference field compares activity patterns
of two different fields connected to its set and reset inputs
producing difference activity pattern.
The neurons of associative field are fully interconnected
with each other forming feedback-type neural network. Associative
neural fields are the basis of this architecture, where the model
of external world is formed and operated. Any information unit of
any complexity (i.e. feature, object, notion, relation, etc.) is
represented not by a single neuron but by some neuron subset
called neural assembly following Hebb [1]. This representation
also preserves the natural likeness between similar information
units in each hierarchical level.
The neurons of associative field possess both the reset/set
inputs and associative ones. If there are no reset/set signals,
neuron output у is defined as
464
 yj = 1, if Sj ≥ t
(1)
yj = 0, if Sj < t
where t stands for neuron threshold, Sj is calculated from
Sj =  i = 1,n yiwij (2)
where w i j is synaptic binary weight of connection from neuron i to
neuron j which may be 0 or 1. The synaptic weight changing
according to our stochastic version of Hebbian learning rule looks
like:
w*ij = wij  (yi  yj  ij), M(ij) = r, (3)
where w*ij and wij stand for binary connection weight from i-th to
j-th neuron before and after learning respectively, yi and yj are
neuron outputs,  - disjunction,  - conjunction,  - negation,
ij is binary stochastic variable that is equal to 1 with the
probability r. The negative reinforcement -1<r≤0 leads to
unlearning:
w*ij = wij   (yi  yj  ij), M(ij) = |r|. (4)
Neural assemblies representing real world patterns are formed in
the associative field in the process of learning. The neuron
activity pattern representing the real pattern is set in the
field, followed by enhancing the connectivity between active
neurons according to Eq.3.
After learning act each connection takes certain value of 0
or 1, but the forming assemblies possess complex structure very
much alike the structure of assemblies in the neural network with
gradual connections. The role of gradual connections is played
here by expectance M(wij) of binary random variables wij. Since
any information unit in the considered neural network is re-
presented by a number (m) of neurons, the neural assembly forms
due to changes in sufficient number (M(r)m2) of connections. The
dependence of connection formation probability between two neurons
versus the number  of their coactivations is given in Figure 1.
The assembly already formed in the network may disappear if
reinforcement is negative.
One of the basic properties of neural assembly is that the
whole assembly is activated provided with activation of its
sufficient part. In the same time the non-assembly neurons are
inactivated. This property appears due to activity regulator
uniformly controlling the thresholds of all the associative field
neurons, that is similar to the regulator discussed in [2].
The regulator provides nearly constant number m of active
neurons within associative neural field. Active neuron number m is
much less than field neuron number (m << n, m  n). The small num-
ber of active neurons stochastically distributed in the pattern
ensures high storage capacity (e.g. [3]). To reduce the number of
active neurons to m in the buffer field, mechanism of another kind
is used [4]. This operation is called normalization.
The specific architecture of associative-projective neural
network (APNN) is constructed from the above neural fields coupled
by projective connections. The assemblies corresponding to items
of various complexity are formed in different associative fields.
For example, in visual recognition the neural assemblies corres-
ponding to roof, wall, window are formed in one associative field,
465
and the assembly representing the whole house is formed in the
other field. Each associative field forms a separate level of the
resulted multilevel architecture. The skeleton of the architecture
consisting of associative fields coupled by projective connections
is shown in Figure 2.
Now let us consider more realistic example of two-level
associative-projective neural network consisting of two associa-
tive fields and one buffer field that is shown in Figure 3. In
this figure and further, neural fields are represented by rec-
tangles, each thick line represents the whole bundle of projective
connections, black arrows represent reset inputs, and white arrows
represent set inputs. We'll demonstrate how real world object
models are formed by this structure. Let the considered objects be
described by a finite feature set F={f1,...,fk}. The neural assem-
blies corresponding to the features are formed in the associative
field AF1. For this purpose all features are sequentially input
to the network by special coding unit CU. The coding unit converts
each input feature fi to some subset of active neurons in associa-
tive field AF1. At the same time positive reinforcement is given
to the field, synaptic connection weights are increased (Eq.3),
and neural assembly corresponding to the feature fi is formed.
Let some object be described by the features fl,f2,f3. To
form the assembly model of the object, its features are input in
turn to the coding unit. The coding unit sequentially activates by
set projective connections the assemblies corresponding to fl,f2,
f3 (see Figure 3. Remember that the assemblies are really coded
not by compact neuron subsets but by stochastically generated
ones). The neuron activity patterns corresponding to these assem-
blies are transferred through projective connections to buffer
field BF where the feature codes are accumulated. After normaliza-
tion only a fraction of active neurons from each feature is
preserved in BF. These are neurons-representatives of features.
Then the obtained pattern is transferred to the associative field
AF2 and the assembly О corresponding to the object model forms
there under positive reinforcement. It is formed from neurons-
representatives of the object features.
The object name coded by subset of active neurons may also be
one of its features (see Figure 4). It permits to activate the
object model by some feature subset as well as by its name N. Let
us now consider how the object name is recognized. If only some
part of the object features is fed to the coding unit, only a
fraction of neurons representing the object will be accumulated in
buffer field BF. As soon as they will be transferred to the AF2
field, the whole assembly of the named object O+N will be
activated there according to the associative retrieving property
of the assembly. Hence the neurons-representatives of the object
name will be also activated. The name itself can be easily
activated in AFN by these neurons.
In the same way the features of assembly activated by its
name can be recalled by their neurons-representatives. The process
of recall is accomplished as follows (see Figure 5). Let the as-
sembly 0+N be activated by its name N in AF2. The activity pattern
of AF2 is transferred to AF1 field. The assembly N is absent in
AF1, so the assemblies of all other object features will get ini-
tial activation from their neurons-representatives. Since the re-
gulator of AF1 field activity maintains the number of active neu-
rons to be equal to that of only one assembly, the assemblies be-
gin competition for activation. It results in activation of one of
the feature assemblies, e.g. f2. This assembly is transferred to
the preliminary cleared buffer field BF and through reset projec-
tive connections will inactivate its representatives in object
assembly in AF2. Then AF1 is cleared and the remaining part of as-
466
sembly (without f2 representatives) is transferred to it.Now only
fl and f2 features will compete for activation. The winner will be
also transferred to BF field and will inactivate its represen-
tatives in the object assembly, and so on. This way the sequential
deciphering of the object assembly to its features is carried out.
Special procedures have been also developed that permit to form
assemblies reflecting the sequence of features presented to the
input and to retrieve this sequence in the deciphering process.
The architecture allows to build adequate hierarchical model
of the external world by addition of extra neural fields connected
by projective connections and the pattern processing procedures
similar to those discussed above.
Sometimes it is necessary to compare activity patterns of two
neural fields. Difference fields are used for this purpose,
allowing to determine the features that differ assemblies from
each other. This property can be used to form relations between
objects, that is necessary to describe scenes, situations, etc.
3. HARDWARE SUPPORT
We consider the described architecture to be universal enough to
provide a uniform base for solution of a wide scope of AI problems
and complicated applications. This allows to create specialized
hardware for supporting this only architecture. Such approach
makes it possible to achieve very high performance of associati-
ve-projective neurocomputers (APNC) that we propose.
Our neurocomputer is a long-word processor oriented for
parallel simulation of a large number of neurons described above.
The processor implements bit operations on long words stored in
word-wide dynamic RAM. The number of simultaneously simulated
input sum S of each neuron (Eq.2) is accumulated in separate
binary counter. The long word processor operates under reduced
instruction set control unit.
For example, our neurocomputer prototype has DRAM of 256K x
256 bit words and simulates 256 neurons of associative field
simultaneously. Since APNN usually contains much more neurons, the
simulation process is only partially paralleled.
This neurocomputer also effectively implements other operati-
ons used in associative-projective architecture. For example,
transferring of neuron activity patterns through projective
connections is accomplished by moving several DRAM words from one
bucket to another. The activity patterns accumulation in buffer
fields is simulated by bit disjunction of corresponding binary
vectors. The random numbers generation for learning in accordance
with Eq.3,4 is accomplished by pseudorandom number generator that
is also simulated in long-word processor, as well as coding of
input information and decoding of output information.
This neurocomputer can implement associative-projective
neural networks with up to 25000 neurons and 50 mln connections.
So far we have implemented neural network with 4096 neurons and 16
mln connections. In the tests one update cycle of this network has
taken 23 ms which corresponds to 700 MCUPS. Neurocomputer is
created as external module for PC AT and is based on standard
middle scale integration components.
Custom chips with 3000 gates implementing long-word processor
section are under manufacturing now. Neurocomputer based on these
chips and 256K DRAM chips will have performance not less than
5000 MCUPS. Using hi-tech electronic components available in the
market today, such as custom chips with 20000 gates and 4MB
dynamic RAM, it is possible to create one-board neurocomputer with
469
100000 MCUPS performance implementing neural networks with up to
200 mln connections. Our latest technical solutions and know-how
permit to develop desktop neurocomputer of 10000000 MCUPS simula-
ting APNN with 5000 mln connections.
4. EXPERIMENTAL INVESTIGATION OF APNN
To investigate the basic properties of APNN, numerous experiments
have been carried out. At first the architecture was simulated by
personal computer, and now neurocomputer prototypes are also used.
A series of experiments on information capacity of
associative field has been carried out [5,6]. For example, the
well-known result [7] that the number of statistically independent
assemblies in the network can exceed the number of neurons has
been observed. The associative field of 4096 neurons could store
and retrieve up to 7000 assemblies each consisting of 64 neurons.
Obviously the assemblies overlap and each neuron may belong to
dozens and hundreds of assemblies, but this does not prevent the
normal operation of the field. The dependence of information
capacity on signal-to-noise ratio in the input pattern and on the
probability of retrieving was also investigated and characteris-
tics important for applications were obtained [8].
The real assemblies formed in the associative field during
information exchange with real environment have rather complicated
internal structure reflecting the structure of objects. The most
general properties of the object form the assembly nucleus with
stronger connectivity than in assembly fringes formed by the
features of individual class representatives. We have simulated
the assemblies with nucleus and fringes for the purpose of
associative field information capacity investigation. In this case
special tests have been performed. The most difficult one is to
retrieve the fringe provided with the assembly nucleus and a part
of this fringe. The experiments have shown substantial decreasing
of information capacity compared to statistically independent
assemblies [9]. Special learning rule was proposed to increase the
stability of fringes.
A number of experiments has been carried out to test the as-
sociative field with "diluted" (not full connected) weight matrix.
The problem was to represent diluted weight matrix in the computer
memory so that the memory size grows linearly (not as the squire)
of the number of neurons preserving the high performance. The
experiments have shown that diluted network operates faster than
full-connected with the same number of connections, and stores
more assemblies with comparatively large size (m > n neurons).
To test the functioning of multilevel hierarchical associa-
tive-projective neural network, the experiments with sequences
processing were fulfilled. The neural network contained four
levels with associative field and several buffer fields in each.
Neural assemblies representing letters were formed in associative
field of the lower level. The neural assemblies of the second
level represented syllables, those of the third level represented
words, and word combinations were represented in the fourth level.
The mechanisms of memorizing, recognition and replaying of complex
letter sequences of various length, including branches and
repetitions, were examined in these experiments [8,10].
To provide the interface of APNN with external world, it is
necessary to transform the input information to n-dimensional
binary neuron vector. For robustness, reliability, enhanced
information capacity, and normal functioning of APNN the codes
must be distributed, stochastic, sparse (with small and controlled
number of is) and reflect the metrics of the coded parameters. Our
470
experiments [8] have shown that the coding procedures developed by us
meet these requirements. These procedures (e.g. [11,12]) are used
in our practical work.
5. APPLICATIONS OF APNN
To test the efficiency of associative-projective architecture in
applications, the experiments in the following domains have been
carried out: isolated vowel recognition in a single associative
field [11], shape recognition of hand-drawn figures in two-level
network [4], image texture recognition [12], inferencing [13]. The
obtained results allowed us to conclude that associative-projec-
tive neural networks can be effectively used in wide range of
applications and to start elaboration of more complex tasks
including visual quality inspection, handwritten text recognition,
acoustical diagnostics, information search systems. They have also
allowed to develop approaches to APNN solution of such difficult
AI problems as visual recognition (including object and scene
recognition), continuous speech recognition, advanced expert
systems including neural network knowledge base and inference
engine with associative substitution, adaptive robot control in
natural environment (including goal formation, decision making,
action planning) [14]. This material will be published in English
elsewhere, but here we'll briefly discuss the experiments on image
texture classification and approach to visual object recognition.
5.1. Experiments on texture recognition
The first stage of recognition task is feature extraction. We have
considered the following characteristics of the image in the "view
window" as texture features: the brightness histogram, the cont-
rast histogram, the elementary edges orientation histogram. Each
column component of each histogram has been considered as separate
feature (parameter) of texture. We've used 75 parameters.
One-level neural network with 4096 neurons and 16 mln modifi-
able connections has been used for this task. The algorithm of
associative field training and functioning looks as follows. In
the mode of supervised training three phases are repeated:
recognition, unlearning and learning. Coded feature histograms
extracted from each image window are fed to the neural network.
The output vector after one step of network functioning is
compared to vectors representing texture classes (masks). If the
output vector has the largest overlap with the mask of proper
class, nothing is done, and we go to the following image window.
If not, the result is considered wrong and unlearning takes place
that results in decreasing connection number from the input code
vector to the mask of wrongly recognized texture class. Then
learning increases the number of connections from the activity
pattern corresponding to the window under analysis to the correct
mask. In the functioning (test) mode, only the recognition of the
new pictures (that was not shown to the network) is fulfilled, and
the statistics are collected.
We have used for experiments a set of 7 black-and-white pho-
tos of the city streets converted to 200x144 images with 36 grey
levels. The images were scanned by 16x16 window with the step of 8
pixel along both axis from the top-left to the bottom-right
corners (the total of 24x17=408 window positions in each picture).
We chose 5 texture types: the sky, the tree crone, the road,
vehicles, tree trunks and polls. First the pictures were marked by
the teacher. Each window position was marked only by one texture
471
type, even if there were several texture types in it - as the
teacher chose to mark it.
A number of experiments has been carried out using various
parameters. First the relative rate of learning and unlearning was
adjusted to provide convergence. Then we have performed experi-
ments to investigate the process of convergence given various
values of initial noise in weight matrix: 1) 20000 random ones and
2) 200000 random ones. The network has been trained on two
pictures until convergence. The convergence curves (averaged over
20 convergence acts) were very much alike in both cases, as the
following table shows:
error number of each picture presentations
rate 1) low noise 2) higher noise
1.2% 38 40
0.6% 60 55
0.0% 76 80
The average number of errors (and corresponding learning-unlear-
ning acts) till convergence was 1700.
The results of the network function on 5 new (test) pictures
have been also obtained. The final recognition rate on these 5
pictures varied from 85,3% to 88,5% with average rate of about
87%. The convergence dynamics (averaged over 5 convergence acts)
can be seen from the following table:
number of each picture
presentations 1 5 10 15 20 30 40 50 60 85
number of errors:
"training" pictures 124 38 21 16 14 10 5 4 2 0
"test" pictures 127 102 66 53 53 65 54 60 56 53
These results (obtained using APNC neurocomputer) were compared
with experiments on the same pictures using potential function
recognition method [15] implemented by us in PC AT. The
recognition quality was the same, but neurocomputer processed one
picture in 30 sec as compared with 1000 sec obtained by potential
functions. We have also conducted these experiments with
perceptron-like neural network in PC AT. Here the recognition of
single picture was faster (about 200 sec), but the recognition
rate was lower: 82-84%. In perspective associative-projective
neurocomputer will process one picture in several milliseconds.
5.2. An approach to visual object recognition by APNN
Complex objects of the external world can be recognized by type,
shape and mutual location of the segmented texture regions. Recog-
nition of texture type has been touched in the previous Section,
and shape recognition has been considered in [4]. Here we describe
basic relations between the texture regions, that can be easily
determined and used in object recognition, and their encoding.
In the first place let us consider space relations and
relative size of the image regions. For example, if there is a
tree in the picture, its trunk is usually located lower than the
crown and near it. Besides, the trunk area is considerably less
than the crown area in the image. The area calculation of the
various image regions can be easily performed by neurocomputer.
The centres of gravity can be also calculated for the regions as
well as the mutual location of any pair.
Let us consider two texture regions A and B. Then a number of
their relations can be determined: P(A,B) - "the region A is near
the region B" relation, L(A,B) - "the region A is to the left of
472
the region В" relation, R(A,B) - "the region A is to the right to
the region B" relation, T(A,B) - "the region A is above the region
B" relation, B(A,B) - "the region A is below the region B" relati-
on, M(A,B) - "the region A is larger than the region B" relation.
Each of the above relations can be characterized by the
numerical value. The former five relations can be characterized by
relative distances measured as the ratio of the corresponding
distance between centres of gravity to the square root of region A
area. The value of M(A,B) can be determined as the ratio of the
region A area to the region В area. To encode these relations, it
is necessary to generate the bit masks for each of them and for
their values. The bit mask is stochastically generated and then
permanently fixed binary vector with known number of ones
corresponding to active neurons. Then bit conjunction of the
relation mask with the value mask of this relation is performed.
Certainly, the above relations are not the only possible relations
between the texture regions, that are used for object recognition.
Thus we have considered encoding of relations between any two
texture regions. To describe the object, it is necessary to
indicate the specific regions that are characterized by these
relations. The example of APNN for relation encoding and object
recognition is presented in Figure 6.
Name
Figure 6. APNN for visual object recognition.
Texture recognition module TR and shape recognition module SR
possess complex structure including several neural fields. Rela-
tion extraction module RE is destined for spatial relations
extraction between uniformly textured regions and may be realized
in neurocomputer as well as in front computer.The obtained spatial
relations are coded in encoding unit CU1. Buffer fields BF1, BF2,
BF3 are for accumulating and normalizing of code representing the
473
object and its name. Object name input module N1 and encoding
module CU2 are for input and encoding of the object name. Name
input may be realized by a number of ways, e.g. by keyboard input.
Associative field AF1 is for formation of neural assemblies cor-
responding to objects in the image, and for their recognition in
operation mode. Decoding module CU3 is for decoding of the recog-
nized object names.
The described system operates in the following way. The image
is fed to the texture recognition module, where the uniformly
textured regions are determined. The neural pattern corresponding
to the region texture type is transferred to the buffer field BF1.
The "map" of the region (i.e. its binary image in the field of
vision) is transferred to the shape recognition module SR and
spatial relation extraction module RE. The assembly of shape most
resembling the shape of the region under processing is activated
in the SR module. This assembly pattern is transferred to the
buffer field BF1, where it unites with texture pattern by bit
disjunction. The obtained code is normalized and transferred to
the buffer field BF2. Then the texture recognition block starts to
determine a new uniformly textured region, that is neighboring the
previous region.
As soon as the new region is determined, its map is
transformed to SR and RE modules. The sequential extraction of the
spatial relations is accomplished in RE, then they are coded in
CU1 and transferred to BF1. This field accumulates the codes of
spatial relations and normalizes them. The remaining code is
transferred to BF2 where it unites by bit disjunction with the
first texture region code. There also exists certain mechanism for
representation of directed relations in the code.
The recognition of the second region shape is then accompli-
shed in SR module. The texture and shape codes are fed to the
field BF1. In the same manner as before the accumulated code is
normalized. The resulting code is input by disjunction to BF2.
The code accumulated in BF2 is then normalized and transferred to
BF3. Meanwhile the system begins the analysis of the following
pair of object regions.
After looking through all the texture regions of the object
the buffer field BF3 contains the code of the object image. The
code consists of Regionl-Relation-Region2 triplets. The code is
normalized and superimposed by bit disjunction with the object
name code input from N1 and CU2 modules. Then the code is trans-
ferred to AF1 where the assembly representing the object forms.
The procedure of the object recognition is constructed in the
same way, except for the object name code is not input to BF3.
This code is associatively retrieved in AF1 and is then decoded in
CU3. Some peculiarities arise in the process of recognition. It "is
not known in advance what area is occupied by the object, in the
field of vision, so when the code is formed originally, the
features of the outside objects may be found in the buffer field
BF3, and at the same time not all the features of the object under
recognition will be there. Three variants of recognition may
occur, i.e. the correct object is recognized, the incorrect object
is recognized, no object is recognized (no assembly is retrieved
in the associative field).
Let us consider the last case. Since the neural network can
not propose any hypothesis about the object, we must move the
analyzed region to the other part of the image or widen it to find
the additional features.
If some assembly is activated in the associative field, it is
considered as the hypothesis about the corresponding object
presence in the field of vision. This hypothesis must be verified.
For this purpose the components of the activated assembly must be
retrieved. The details of this process were considered in Sectio-
2. This process must be conducted down to the level of texture
assemblies and spatial relation assemblies. Then more precise
474
View publication stats

determination of the texture regions belonging only to the object
under analysis may be accomplished, and the image area occupied by
the object is specified. The assembly may contain the textures
that were not in the field of vision originally. The deciphering
allows to initiate the aimed search of the missing textures on the
basis of the component texture features as well as their supposed
location relative to the already discovered texture regions.
This additional analysis must be concluded with the second
attempt of object recognition. These iterations may be continued
until the precise analysis in the lower level brings new results.
Now two outcomes are possible: either the composition of the
discovered texture regions fits the set of the textures in the
recognized assembly or some texture regions are absent. In the
latter case there may be different reasons of the discrepancy. One
of these may consist in our examination of the particular object
that possibly does not contain all the features of the typical
pattern that is formed as a neural assembly. The second reason may
be that object is partly hidden by another object. This reason may
be discovered if the recognition of a new object is accomplished
in the places where the missing texture regions must be located as
a result of deciphering. If well-recognized object is discovered
in this place, it is highly probable that it hides the missing
texture regions. Assumption of object shadowing is also probable
if the missing texture regions are typical to this object class.
Thus the object recognition requires iterative procedure
including transition from the lower hierarchical levels to the
higher ones as well as in the opposite direction. This procedure
permits to verify the recognition correctness and to specify the
object and its parts position, to tell the object from the
background, and to determine if it is shaded by another object.
[5] Rachkovskij D.A. On numerical-analithycal investigation of
neural network characteristics. Neuron-like networks and neurocom-
puters. Kiev, Inst.Cybern.Ukrain.Acad.Sci. - 1990 - P 13-23 ( i n
Russian).
[6] Baidyk T.N., Kussul E.M., Rachkovskij D.A. Numerical-analyti-
cal method for neural network investigation. Proc. Int Symp on
Neural Networks and Neural Computing. Prague, Czecho-Slovakia
Sept. 10-14, 1990. - P.217-219.
[7] Willshaw D.J., Buneman O.P., and Longuet-Higgins H.C. Nonholo-
graphic associative memory. Nature. - 1969. - 222. - P.960-962
[8] Rachkovskij D.A. Development and investigation of multilevel
assembly neural networks. Ph.D.Thesis. - Kiev, 1990. (in Russian)
[9] Baidyk T.N., Kussul E.M. Structure of neural assembly (Sub-
mitted to NIPS'91).
[10] Kussul E.M., Rachkovskij D.A. Multilevel assembly neural
architecture and processing of sequences. (Report presented at the
International Workshop "NEUROCOMPUTERS and ATTENTION", Moscow
Sept 1989. Proceedings volume will appear in the series "Proceed-
ings in Nonlinear Science", Manchester University Press)
[11] Rachkovskij D.A., Fedoseeva T.V. On audio signals recognition
by multilevel neural network. Proc. Int. Symp. on Neural Networks
and Neural Computing, Prague, Czecho-Slovakia Sept 10-14, 1990.
- P.281-283.
[12] Kussul E.M., et.at. On image texture recognition by
associative-projective neurocomputer. (Submitted to ANNIE'91).
[13] Experiments with our novel analogy-based inference have been
carried out by A.M.Kasatkin and L.M.Kasatkina.
[14] Kussul E.M. Associative neuron-like structures (in Russian)
The book will be publised by Naukova Dumka, Kiev, in 1992.
[15] Eiserman M.A., Braverman E.M., Rosonoer L.I. Potential
function method in machine learning theory. - Moscow, Nauka, 1970.

6. CONCLUSION

We have described neural network architecture developed especially

for solution of difficult AI problems. Experiments in recognition
of vowels, textures, shapes, processing of sequences and logical
inference conducted on neurocomputer prototypes have demonstrated
efficiency of the proposed architecture in certain aspects of
these problems. The uniform approach to coding and processing of
information in different hierarchical levels and modalities allows
to unite the developed fragments in the frames of single neural
network structure and to obtain the system that will be able to
solve complicated problems that are insoluble today by other
neural network architectures and traditional approaches. The
present architecture permits to create non-expensive and high
performance hardware support, that makes it attractive from angle
of perspective applications. Such hardware is under development
now. As new hardware support will be ready we intend to expand the
range of applications.

REFERENCES
[1] Hebb D.O. The Organization of Behaviour. - N.Y.: Wiley, 1949.
[2] Braitenberg V. Cell assemblies in the cerebral cortex. Lect.
Notes Biomath. - 1978. - 21. - P.171-188.
[3] Palm G. On associative memory. Biol.Cybern. - 1980. - 36. -
P.19-31.
[4] Kussul E.M., Baidyk T.N. Development of neural network archi-
tecture for recognition of object shape in the image. Automatika.-
1990. - N5. - P.56-61. (in Russian).

475 476