of interest:The study of neurodynamics may follow one of two routes, depending on the application
in which the neural network model has a deterministicbehavior. In mathematical terms, it is described by a set of
nonlinear differential equations
that define the exact evolution of the model as a function of time (Grossberg,1967; Cohen and Grossberg, 1983; Hopfield,
in which the neural network model is perturbed by thepresence of noise. In this case, we have to deal with
stochastic nonlinear differential equations,
expressing the solution in probabilistic terms (Amari et al., 1977; Amari,1990; Peretto, 1984). The combination of stochasticity and nonlinearity makes thesubject more difficult to handle.In
chapter we restrict ourselves to deterministic neurodynamics. The neural network model used in the study
based on the
of a neuron that was derived inSection 13.2. As mentioned already, an important property of a dynamical system is thatof stability. To understand the stability problem, we need to start with some basic conceptsunderlying the characterization of nonlinear dynamical systems? which is how we intendto pursue the study of neurodynamics.
Organization of the Chapter
The material presented in this chapter is organized as follows. In Section 14.2 we brieflyreview some of the basic ideas in dynamical systems. This is followed by considerationof the stability of equilibrium states of dynamical systems in Section 14.3; here weintroduce Liapunov’s theorems, which are basic to the study of the stability problem.Then, in Section 14.4, we describe the notion of
which plays an important rolein the characterizationof dynamic behavior of nonlinear systems.
is naturally followedby a discussion of
in Section 14.5.Having familiarized ourselves with the behavior
dynamical systems, we move onto present general considerations of neurodynamics in Section 14.6, where we describea neurodynamical model of recurrent networks. In Section 14.7 we discuss recurrentnetworks in a general neurodynamical context.Then, in Section 14.8, we consider the continuous version of the Hopfield network,and define the Liapunov function for it. In Section 14.9 we state the
dynamical systems, which includes the Hopfield network (and otherassociative memory models) as special cases.
Section 14.10 we study the applicationof the Hopfield network as a content
addressable memory, and
revisit some of theissues discussed in Chapter
In Section 14.11 we consider another recurrent network, called the brain
model (Anderson et al.,
and study its dynamic behavior
an associativememory.In Section 14.12 we consider the recurrent back
propagation algorithm and the associ
ated stability problem; the algorithm is so
called because the learning process involvesthe combined use of (1) a neural network with feedback connections and (2) the
propagation of error signals to modify the synaptic weights of the network.
a graphical, easy-to-read treatment
Abraham and Shaw (1992). For
nonlinear dynamical systems with an engineering bias, see
(1981). For a mathematical treatment
nonlinear dynamical systems in a more abstract setting, see
and Hirsch and
The two-volume treatise by
Jackson (1989, 1990)
perhaps the most complete treatment
the subject, providing a historical perspective and a readable account
nonlinear dynamics that integrates both classical and abstract ideas.