Section 6

Network Architectures

21

The analysis of the dynamic behavior of neural networks involving the application of feedback is unfortunately complicated by the fact that the processing units used
for the construction of the network are usually nonlinear. Further consideration of this
important issue is deferred to the latter part of the book.

NETWORK ARCHITECTURES
The manner in which the neurons of a neural network are structured is intimately linked
with the learning algorithm used to train the network. We may therefore speak of learning algorithms (rules) used in the design of neural networks as being structured. The
classification of learning algorithms is considered in Section 8. In this section, we focus
attention on network architectures (structures).
In general, we may identify three fundamentally different classes of network
architectures:

(i)

Single-Layer Feedforward Networks

In a layered neural network, the neurons are organized in the form of layers. In the simplest form of a layered network, we have an input layer of source nodes that projects
directly onto an output layer of neurons (computation nodes), but not vice versa. In
other words, this network is strictly of a feedforward type. It is illustrated in Fig. 15 for
the case of four nodes in both the input and output layers. Such a network is called a
single-layer network, with the designation single-layer referring to the output layer of
computation nodes (neurons). We do not count the input layer of source nodes because
no computation is performed there.

FIGURE 15 Feedforward network

with a single layer of neurons.

Input layer
of source
nodes

Output layer
of neurons