that modularity as a computational technique
the key to understanding complex tasksperformed by artificial neural networks. Unfortunately, the scope of
knowledge of this important subject is rather limited at the present.
chapter on a very special
of modular networks should therefore be viewed as a good beginning.
The main body of
chapter on modular networks is organized as follows. In Section12.2 we formally define what we mean by a modular network, and discuss the implicationsof modularity.
Section 12.3 we describe an associative Gaussian mixture model for aspecific configuration of modular networks. This is followed by the derivation of astochastic-gradientlearning algorithm (based on maximization of a log
likelihoodfunction)for the network, which we do in Section 12.4. In Section 12.5 we extend the concept of modularity by developing a hierarchical structure of modular representations. In Section12.6 we describe an application of modular networks in control. The chapter concludesin Section 12.7 with a summary of
properties of modular networks and some finalthoughts on the subject
modular network is formally defined as follows.’
neural network is said to be modular
by the network can be decomposed into two or more modules (subsystems) that operate
distinctinputs without communicating with each other. The outputs
the modules are mediated by an integrating unit that is not permitted to feed information back to the modules.
particular, the integrating unit both
decides how the outputs
the modules should be combined to form the
the system, and
decides which modules should learn which training patterns.
Modularity may therefore be viewed as a manifestation of the
divide and conquer,
which permits us to solve a complex computational task by dividing it intosimpler
and then combining their individual solutions.
modular network fuses supervised and unsupervised learning paradigms in a seamlessfashion. Specifically, we have the following.
exemplified by an external “teacher” that supplies the desiredresponses (target patterns) needed to train the different modules of the network.However, the teacher does not specify which module of the network should produceeach desired response; rather, it is the function of the unsupervised learning paradigmto do this assignment.
exemplified by the modules “competing” with each otherfor the right to produce each desired response. The integrating unit has the role of “mediating” among
different modules for
right.Consequently, the modules of the network tend to
by learning different regionsof the input space. However, the form of competitive learning described here does notnecessarily enforce the specialization; rather, it is done naturally.
the competition,roughly speaking, the winner is the module whose output most closely matches the desired