INTELLIGENT

4 SYSTEMS
 Architectures for Complex Behaviour
Inspired in the Nervous System

Roberto Moreno-Diaz 1 and Jose Mira Mira 2

Centro Internacional de Investigacion en C. de la Computacion Universidad de Las

PMmas de Gran Canaria, Canary Islands, Spain.
2 Dpto. de Inteligencia Artificial UNED, Madrid, Spain.

A b s t r a c t . Generalized robotic systems are probably one of the more

complete paradigms of natural behaviour, so that they can properly be
consider to show artificial behaviour in the higher sense of the term. A
generalized robotic system is here understood as that artificial system
capable of sensing the environment, perceive it and interprete it in terms
of more or less general models of the environment and of itself; capable
also of computing appropriate strategies of action, by planning and by
finding plausible action methods. And capable, finally, of firing sequences
of motor and effectors actions, having ways of controlling said actions.

1 Introduction
Such a complete system is still a desideratum nowadays though not so highly
sophisticated types of artificial behaviour can be, at least theoretically, obtained
by piecing together different components from the fields of robotics and artificial
intelligence, and by using the inspiring help obtained from naturM systems.
One of the central issues to this goal is how sensing, perception and action
are integrated in non trivial manners. Though this is not a recent goal, modern
artificial intelligence tools are shiding some light to built practical devices. More
than twenty years ago, members of a research program carried on at the Instru-
mentation Laboratory of the Massachussets Institute of Technology [7], though
initially dealing with artificial visual perception, soon realized that many artifi-
cial perceptual problems were not comprehensible unless the perceptual system
was considered to be integrated within a king of robotics system. Thus, the
questions of the optimal use of visual data were treated in conjunction with the
use of said information to converge into a "mode of action", by some type of
cooperative decision process [8] [9]. We dissect here a frame for such integration,
which is general enough to be non- trivial, but that can be -and is being- used
to design practical robotics systems which are controlled and commanded by
planning subsystem based on sensory -mostly visual- data.

2 A General Structure

A simplification of a very general view of the nervous system in terms of computer

analogues proposed by McCut!och and Sutro [9] is shown in figure 1. Though
 354

this really simplifies their original proposal, it has the considerable advantage
that each one of the large components shown has very concrete counterparts in
modern artificial intelligence and robotics.
First, the overall system presents a set of "modes of behaviour" that mimic
the accepted model of behaviour of the vertebrates [2] [3]. The selection of a
particular mode is performed by the command and control system, based mostly
in present sensorial information (S.D.) and the status of the system. An external
input (E.I.) is allowed from the external world (in practice, it comes from an
operator's console) to modulate the selection of a mode.
Information concerning the selected mode (M) is sent to the sensors, which
are to be tuned to optimize the data acquisition pertinent to the mode of ac-
tion. It is also sent to the component labeled Files of World Representations,
in which the appropriate model of the environment and of the system in the
environment is selected to be sent to the component labeled Planning in Present
World Representation. Finally, the selected mode commands and controls the
process of stablishing goals according to the mode, the process of planning and
the execution of the plan, by taking into account continuous highly processed
sensory data (S). Actualized world representations are sent back through W.R.
lines when the mode changes. There are also direct connections between sensors
and actuators (line R) which are equivalent to reflex paths. Line E provides for
high level instructions to the effectors according to the plan of action, which are
to be decoded into concrete motor-effector actions.
An appropriate computer architecture to embody such a system, according
to present day technologies, is shown in Figure 2. There are two types of special-
ized processors concerning the sensory data and concerning the effectors, which
hung on the corresponding buses. Command and control, as well as planning is
performed by the Kernel, while computations corresponding to models of the
environment are performed in the unit labeled C.M.E. (Computer Models of the
Environment).

3 Command and Control

The basic function of a command and control system is to commit the whole
system to one overall mode of behaviour belonging to a not very large set. This
is what enables it to behave as an well-integrated unit instead of a loose connec-
tion of separate sensors, effectors and processors. In this sense, a command and
control computer is a close paradigm to the operation of the reticular formation
in vertebrates [4]. Any modes of behaviour is incompatible with any other. Some
general properties can be stablished for such a computer. First, it receives rel-
atively unprocessed information from all of the sensors situated in sensory and
effector sub-systems. Second, it gives signals, which control, tune and set the
filters of all external inputs. In McCulloch words [2], "this is the structure that
decides what to look and having looked, what to heed". It also controls all the
information flow from and to higher level computers.
 355

From a structural point of view, the command computer must have a modular
architecture, or, at least, it must simulate it. The basic idea is that a set of
computing units (C.U.) is such that each computing module receives information
only from a reduced part of the overall, little processed sensory inputs.
Each computing unit is capable of both general diagnostics about overall
input situations and of specialized diagnostics according to the values of a concret
subset of the input lines.
A crucial point is that a consensus of the diagnostics -which corresponds to
the selection of a single mode of behaviour- must be reached by the computing
units in a relatively short time. This requires a very strong crosstalk among
the computing units, which is a peculiar feature of the so-called cooperative
processors [4]. There are two basic properties of the computing modules that
can be stated easily by means of the terminology common in expert system.
In fact, we can look at the computing units as if they were simplified experts
systems working on their own databases and with their own inference engines on
their specialized domain of sensory inputs. But they are capable also of giving
up before the evidence in diagnostics by other units, which show to have more
relevant information for the case. This "giving up" must be understood in the
sense of a recruiting of the rest of the modules by those having more confidence
about their diagnostic. As it was stated by McCulloch some twenty years ago [2],
modules having the information pertinent to the case "cry louder", and doing
so, they recruit the rest. The result of this strong erosstalk is that the system
converges into one mode, in the sense that practically all the units decide the
same mode of behaviour, though with perhaps different degree of confidence.
Modularity and division of expertise, with overlapping, among the computers
units are the two basic features of a cooperative processing system. Also, appro-
priate crosstalk rules are the necessary addendum to achieve convergence. This
architecture is supposed to provide for two main goals: first, to speed up the
decision process by which a mode of behaviour is selected; second, the system
is supposed to present high reliability, in such a way that it will arrive into an
appropriate eonsensed mode, even when some of the expert units are destroyed.
This second aspect, that is, the reliability intrinsic to distributed expertise,
precludes any decision based upon a single majority organ, because its malfunc-
tion will imply total inoperativily. That is, the conclusion that a consensus has
been reached cannot be the output of any special testing unit receiving its in-
puts from the expert units. Instead, the decided modes must be appropriately
labeled according to their procedence to prevent mixing, and be present in a
non-computing structure, that is, a set of wires -or axons- or, in other words, in
a kind of decision bus.
From this, it becames clear that reaching a rapid consensus in the mode of
behaviour at the command and control computer is absolutely necessary for the
rest of the system to operate coherently, because, otherwise, the various higher
and lower subsystems to be controlled, will have a high probability of picking up
operation instructions from the decision bus, which belong to different exclusive
modes of behaviour, such that a kind of neurotic situation will be created.
 356

In sum, the role of the command and control computer is to sense crude
sensorial data, and to decide modes of behaviour sending them to the decision
bus, through a strong crosstalk among units to converge into a single mode, so
that coherent behaviour is secured.

4 Multisensorial Environment Representation

There are two basic ways to structure multisensorial information which, in turn,
admit different levels of representation, form geometric to highly symbolic, at
least for artificial systems. These two ways have a correlate with the finality
of the representation which may tend to be optimal for discriminating among
environment patterns or to be representation to optimize the acquisition of clues
for actions. These correspond to:
a) Integrated representation, both at low levels of acquisition and a high level
of the processing of sensory data.
b) Step by step representation, in which integration only occurs at high levels
-that is in symbolic structures.
In other words, and putting aside for the moment all natural systems, we
may either represent the sensed environment by means of a multidimensional
space where all sensory modalities are present with their own resolution at low
level, while all high level processing is performed directly in this space. Or we can
construct a high level intersensorial representation space by previously extracting
properties; classifying and labeling each sensory modality separately. These two
options are illustrated in figure 4 (a) and 4 (b).
These two possibilities seem to coexist in the highly parallel computing struc-
tures of natural systems. Thus, when trying to explain the strong discriminating
power of the nervous system at high recognition and perception levels, it seems
logically necessary to admit that a kind of integrated representation is present,
because low level, almost primarily sensed clues -like a pitch or a colour- are
definite clues to identify high level situations. And also, there are very fast re-
sponses of natural systems before key overall internal-external situations that
cannot be explained if elaborate computation in multisensorial representation
spaces where required. In any case, it seems that the two coexisting mechanisms
have a type of principle of constancy, in the sense that increasing one of them
implies decreasing the other. A more detailed treatment of said mechanisms was
presented in [5].
The above duality is, in fa~t, embodied in the proposed general structure of
figure 1. Thus, the command and control computer will operate on data which
follow the representation scheme of figure 4 (b), while the Files of World Rep-
resentations, and, subsequently, the Planning Systems, operate according the
scheme of figure 4 (a). This implies that, when setting the mode of operation,
different sensory modalities are taking separately the task of extracting medium
level descriptors, which are necessary for the operation of the command and
control computer. But, once the mode is decided upon, representation and plan-
ning structures shall take again the sensory data without much processing to
 357

"navigate" in a multisensorial space where to perform processes, from low levels

to high levels.
This interpretation incides upon the old problem of how the nervous system is
capable of sharing high level computing structures after depresion or destruction
of primary sensory channels [4]. T h a t is, the so called intersensorial transforma-
tions can only occur in an integrated representation space such of that in the fire
4 (b). Therefore, the command and control computer in natural systems have
not the ability to use directly the expertise of computing units acting upon sen-
sory data which are depressed, while in the planning and representation spaces,
this is a possibility. In other words, there seem to be two different mechanisms
for attaining reliability and speeding up a decision a n d / o r recognition. T h a t is,
the command and control system operates with expert units which receive very
restricted data, while the planning and higher structures have general purpose
units which may work on almost row data form different sources.

5 Intersensorial Transformations

Intersensorial Transformations refer to transformations between different sensory

modalities such that patterns belonging to one sensory modality could be, under
circumstances, handled and "recognized" by computing structures which belong
to a different sensory modality.
The first requisite is to transform the original sensory data to cuasi-geometrical
representation spaces. In artificial sensory systems we can think of a variety of
instances of how a representation space can be constructed, for visual or for
other input. From each representation, space mechanisms acting as "operators"
on the space, provide for outputs carrying some type of decision or classifica-
tions, which are required for the overall diagnosis of the sensory situation. This
overall output enters a "sensory integration" process, to produce that overall
diagnosis, by applying some decision rules where the "blanks" in the rules are to
be filled with the data provided by each sensory modality. Intersensorial trans-
formations "act" among the representation spaces, that is, interconnect them.
We shall make variations on this.
As for the central nervous system, it is clear that specific sensory pathways
became multifunctional as they proceed upwards toward more central areas, in
such a way that structures which are specific can, under circumstances be used
by different sensory modalities. This is so in emergency situations, e.g. under
traumatic lesions. In other words, functional specificity is less dogmatic and
more circumstantial as more central zones are considered in a sensory pathway.
This characteristic must be taken into account when the high degree of reliability
of nervous tissue is to be explained.
A simplified representation of various medium level sensory cortex functions,
as they are normally accepted in brain theory, is precisely that shown in figure
4.b. Here a typical process of sensory integration is represented, which goals are
the diagnostics of an overall external sensory situation.
 358

Sensory data form each modality is preprocessed in specific pathways, before

they reach the cortex, in a way such that a translation to a specific symbolic lan-
guage takes place. This language is the one understood by the processors at the
primary sensory cortex. It is important to point that the messages being trans-
mitted to the primary sensory cortex are already represented in a language which
probably is much more complex that the usual pulse code normally accepted in
neurophysiology. Primary sensory cortex classifies and labels the corresponding
sensory apparitions. This classification and labeling are a consequence of map-
ping in representation spaces and of applying classification and decision rules,
which are either acquired by learning or built-in. One of the implicit properties
which are normally ascribed to primary sensory cortex processors is their almost
total specificity, in a way such that practically no possible intersensorial trans-
formation takes place, but just a posterior multisensorial integration to converge
in the overall diagnosis.
This is a consequence of admitting that the crucial mechanism to attain
diagnoses convergence is a high level multisensorial integration, with no previous
intersensorial talk. From the behavioral point of view, multisensorial integration
can be thought to be directed towards two essential goals: a) to accelerate and
optimize in time and resources, the diagnostic and identification process, by
means of the use of intersensorial clues (that is, labels which, by learning or
instinct, are associated to belong to a same extern-internal global situation), b)
To separate classes of situations, which are not separable by means of a single
sensory modality, so increasing the discriminating power of the sensory system.
The diagram of figure 4.b fulfills the requirements corresponding to the above
goal. Obviously, this diagram can be sophisticated by adding a series of refine-
ments in the processors corresponding to primary sensory cortex and to associa-
tive or integrating cortex, such as a sensory model of the environment, expressed
in a more or less abstract language. Also, the existence of centrifugal (feedback)
pathways can be included to adapt the resolution and subsequent computational
costs of the processors. The concepts of intersensorial transformation permit,
however, a qualitative change in said diagram.
If we assume that a lesion in the specific pathways happens, according figure
4.b., the processors corresponding to the primary sensory cortex will became
out of use, since there are no inputs. However, the experiments of J. Gonzalo [7]
show that is not strictly the case. That is, the practical absolute specificity of the
processors in the primary sensory cortex tends to disappear in the case of lesion
of the specific afferents, after a learning process. Also, the drawbacks, which re-
sult from primary sensory cortex lesions, can be diminished after training, such
that the eliminated function is partly assumed by the remaining cortical struc-
tures. The above can, in principle, be expressed in terms of a global behavioral
goal, as it was done for multisensorial integration. The assumption is as follows.
There exists and adaptive sharing of the specific structures of different sensory
modalities, such that, after learning, intersensorial information can be utilized to
increase reliability and to tend to optimize the use of the structures which remain
after lesions or when there is a low information input to a sensory pathway.
 359

A possible systems solution is diagrammed in Figure 5. Intersensorial trans-

formations IT occur between the processors specific to each sensory modality.
There are two cases in which, computerwise, we may think of intersensorial
transformations coming into play: 1) When there is an overflow of sensory infor-
mation from a sensory pathway (A) to their more central specific processors, and
there are dimensions which are free (non utilized) in the representation spaces of
other sensory modalities (B, C). This overflow may appear as a consequence of a
traumatic reduction in the specific processors functionality of (B, C). The trans-
formation proceeds from (A) to (B, C). 2) When, because a traumatic lesion or
circunstancial effects, there is a low sensory flow from the sensory pathways of
a modality (A), which free dimensions in their representation space. These free
dimensions can be occupied by data through transformations from other sen-
sory modalities (B, C). And also, there are computerwise possibilities to define
intersensorial transformations among representation or "features" spaces. But
in purity, when talking about brains, and not about robots, the above must be
taken as metaphors.

6 Planning

Planning takes place in a world representation scenary or stage, which corre-

sponds to the selected mode of behaviour. It is useful to consider that this may
be performed at various levels of sensory data elaboration. The question of gen-
erating plans for complex problem solving as a matter of artificial intelligence
techniques, suggests two practical-artificial problem domains that are of imme-
ciiate use. First, there is the problem of moving solid shapes through a restricted
world, and second, there is planning in blocks world [1] [6].
The first problem domain is very close to that of navigating a robot by
avoiding obstacles, and it is based on a scenary whose representation may be
geometric -like, having low symbolic content. The second problem domain may
contain symbolic scenaries and the planning be performed at higher levels.
This suggests a first dichotomy in planning that can be cascaded under coin-
mand and control of the selected mode of behaviour, as it is illustrated in Figure
6.
Planning on the low-level scenary can be formulated in quasi-geometric terms.
The scenary, itself is, in most of cases, determined by a set of an independent
variables, defining a "space", plus a number of restricted zones. These restrictions
are volumes Vi, with decision rules defining the membership to Vi for any point
of the space. Objects are sets of points P1...PN not belonging to Vi which are
subjected to K relations RK (Pi). Generating a plan consists essentially in finding
trajectories for Pi, in agreement with restrictions and relations, under a "goal
field".
At higher levels, planning can be stated in a similar way, by appropriately
changing the level of signification of the concepts, and subsequently defining the
rules for handling scenaries, objects, goal fields and trajectories.
 360

7 Conclusion

The integrated view of a system displaying non trivial artificial behaviour re-
quires the inclusion, at least, of the general components described in precedent
sections. The proposed general structure is concrete enough to be of practical
realizability. Development of the details of the structure and functions of the
components as well as the communications is a research under way, to be ap-
plied to practical visual robot systems.

References
1. Brooks, R., Lozano-Perez, T.: A Subdivision Algorithm Configuration Space for
Findpath with Rotation. Proc. Eight IJCAI, A. Bondy, de. Los altos, Calif.: W.
Kauffam. (1983) 799-806
2. Kilmer, W., McCulloch, W.S.: The Reticular Formation Command and Control Sys-
tem. In: Information Processing in the Nervous System. K.N. Leibovic, ed. Berlin,
Heidelberg, New York: Springer. (1969) 297-308
3. Kilmer, W.: A Global Imhibitory Model for Deciding Modes of Attention. In Brain
Processes, Theories and Models. R.Moreno-Diaz, J.Mira-Mira, eds. The MIT Press.
Cambridge MA. (1996) 125-133
4. Mira, J., Delgado, A.E., Manjaros, A., Ros, S., Alvarez, J.R.: Cooperative Processes
at the Symbolic level in Cerebral Dynamics: Reliability and Fault Tolerance. In
Brain Processes, Theories and Models. R. Moreno-Diaz, J.Mira-Mira, eds. The MIT
Press, Cambridge MA. (1996) 244-255.
5. Moreno-Diaz, R., Mira, J.: Conceptos para una teoria de la integracion multisenso-
rial. Proc. II Simpos. Nac. IFAC. Zaragoza, Spain: Universidad (1984) 227-232
6. Nilson, N.J.: Principles of Artificial Intelligence. Palo Alto, Calif." Tioga. (1980)
7. Sutro, L.L., Warren, R.E., Moulton, D., Whitman, C., Zeise, F.: 1963 Advanced
Sensor Investigations I{-470. Instrumentation Laboratory. Cambridge, MASS: MIT.
(1964)
8. Sutro, L., Kilmer, W., McCulloch, W.S., Catchpole, R., Tweed, D., Blum, J., Peter-
son, D., Morenc~Diaz, R.: Sensory, Decision and Control Systems. R-548. Instru-
mentation Laboratory. Cambridge, MASS: MIT (1966)
9. Sutro, L., Kilmer, W.: Assembly of Computers to Command and Control a Robot.
1{-582. Instrumentation Laboratory. Cambridge, MASS: MIT (1969)