Cellular neural network

Tamás Roska and Giovanni E. Pazienza (2009), Scholarpedia, revision #72952 [link to/cite this
4(12):1519. doi:10.4249/scholarpedia.1519 article]

Hosting and maintenance of this article is sponsored by Brain Corporation.

Curator: Dr. Tamás Roska, SZTAKI, Hungarian Academy of Sciences & Pázmány University,
Budapest, Hungary
Curator: Dr. Giovanni E. Pazienza, SZTAKI, Hungarian Academy of Sciences & Pázmány
University, Budapest, Hungary

Figure 1: Two-dimensional CNN withradius of neighborhood r=1': the red cell has nine neighbors (the eight blue

cells and itself ).

A Cellular Neural Network (CNN), also known as Cellular Nonlinear Network, is an array
of dynamical systems (cells) or coupled networks with local connections only. Cells can be
arranged in several configurations; however, the most popular is the two-dimensional CNNs
organized in an eight-neighbor rectangular grid. Each cell has an input, a state, and an output,
and it interacts directly only with the cells within its radius of neighborhood r: when r = 1, which is
a common assumption, the neighborhood includes the cell itself and its eight nearest neighboring
cells (see Fig. 1). In general, the state of each cell, and hence its output, depends only on the
input and the output of its neighbor cells, and the initial state of the network. By varying the values
of the connections among cells (i.e., its interaction weights), a CNN can present a large number
of dynamics, as proven by Gilli et al. (2002).

Among the numerous applications of CNNs, image processing is probably the most widespread
(vision systems based on CNN computers are commercially available), even though recent
 studies have proved that they can be also used for simulations in fluid dynamics (Kocsárdi et al.,
2009) and statistical physics (Ercsey-Ravasz et al., 2008).
Contents
[hide]

• 1 Standard CNN model

• 2 CNN - Universal Machine and Cellular Wave Computing

• 3 Biological and technological motivations

• 4 References

• 5 Recommended readings

• 6 See also

Standard CNN model

The CNN dynamics is described by a system of nonlinear differential equations. Using the
simplest first-order cell dynamics and linear interactions, the state equation of a cell in
position (i,j) is as follows:

where uij, xij, and yij are the input, the state, and the output of the cell in position (i,j), respectively;

the indices k and l denote a generic cell belonging to the neighborhood of the cell in
position (i,j). All variables are continuous. The set of matrices and the threshold {A,B,z}, which
contains the weights of the neural/nonlinear network, is called the cloning template and it defines
the operation performed by the network. When the values of the cloning template do not depend
on the position of the cell, the CNN is called space-invariant. In this case, the dynamic behavior of
the network depends only on a few parameters; for instance, for a two-dimensional CNN with

a radius of neighborhood r=1, A and B are 3 3 matrices, while is a scalar; therefore, in

total just 19 real numbers determines the CNN dynamics.
Figure 2: Standard nonlinearity for the output equation.

The expression for the output yij is

which corresponds to the nonlinear function in Fig. 2. Note that other functions are also
possible (see Fig. 2 in Chua and Roska, 1993).

Figure 3: Local connections of an edge cell. Observe that three of its neighbors are boundary cells(dashed).

A standard two-dimensional CNN is defined on an M N rectangular array. Therefore, in order

to calculate the state of the cells on the boundary of the array, it is necessary to define the
boundary conditions of the network, as shown in Fig. 3. In the standard model, the boundary
conditions can be:

§ Fixed (or Dirichlet), if the value of the boundary cells is a prescribed constant;
§ Zero-flux (or Neumann), if the value of the boundary cells is the same as the edge cells;
§ Periodic (or toroidal), if the value of the boundary cells is the same as the edge cells on the
opposite side (e.g., top boundary cells have the value of bottom edge cells).
Given a cloning template meeting certain conditions of convergence, the CNN converges to a
steady state which depends on the initial state and the input.
Clearly, the concept of Cellular Neural/Nonlinear Networks is closely related to that of Cellular
Automata: in particular, every binary Cellular Automaton of any spatial dimension is a special
case of a CNN with the same neighborhood size, as proven by Chua (2006). However, the CNN
has input, as well.

CNN - Universal Machine and Cellular Wave Computing

Figure 4: Architecture of the CNN - Universal Machine (click to enlarge).

The potential applications of a Cellular Neural/Nonlinear Network are fully exploited in the CNN-
Universal Machine (CNN-UM), whose architecture is shown in Fig. 4. A CNN-UM (see Roska and
Chua, 1993) consists essentially of a grid of CNN nuclei, which have local memories and basic
local processing capabilities, controlled by a global programming unit. A typical CNN-UM program
is composed of a sequence of operations (logic and analog) defined by cloning templates. Since
each single cell of the network has its own input, state, and output, CNNs can easily integrate
sensing and processing. For example, if the CNN cells have visual sensors, then an image can
be processed directly at the focal plane without requiring any data transfer to a central processing
unit, as instead occurs in standard computers. It has been proved that the mammalian retina
operation can be modeled according to similar principles (Bálya et al., 2002).

A Cellular Wave Computer (Roska, 2003) is an extension of the Cellular Neural Network -
Universal Machine in which input and output are flows and elementary instructions are solutions
of partial difference-differential equations. The philosophy of this paradigm is completely different
from that of a standard digital computer. Complex problems, like a typical reaction-
diffusion equation, become trivial in this architecture, since the problem is solved by a single
instruction.

It is possible to prove rigorously that both the CNN-UM and the Cellular Wave Computing
machine are universal in the Turing sense.
Biological and technological motivations
Cellular Neural/Nonlinear Networks were conceived by Chua and Yang (1988) as a particular
model of Neural Networks. Nevertheless, the recent advances in Neuroscience have confirmed
that CNNs can model the working principles of many sensory parts of the brain. For example, the
Cellular Wave Computing paradigm is based on several of the key properties of brain-like
systems, such as:

§ Continuous-time continuous-valued (analog) signal arrays (flows)

§ Several two-dimensional strata of analog processors (neurons)
§ Vertical interconnections between a few strata of neuron processors
§ Typically, mainly local or sparse global interconnections
§ Integration of sensing and processing
§ Spatial-temporal active waves
§ Events are patterns in space and/or time

These features cannot be found in current standard computers which work according to the
principles conceived by Von Neumann – logic machines acting on discrete-valued data
programmable algorithmically by using logic sequences of arithmetic and logic operations – which
were in agreement with the model of neurons as threshold logic envisaged a few decades ago.

Furthermore, technology has changed dramatically in the past few years: nowadays, it allows the
integration of multiple, even thousands, processors on a chip. The high working frequencies of
such processors force to use mainly local communications which have the further advantage of
reducing the wiring required and power consumed.

References

§ Bálya, Dávid; Roska, Botond; Roska, Tamás and Werblin, Frank S. (2002). A CNN framework
for modeling parallel processing in a mammalian retina. International Journal of Circuit
Theory and Applications 30(2): 363-393.
§ Chua, Leon O. and Yang, Lin (1988). Cellular neural networks: theory. IEEE Transactions on
Circuits and Systems 35(10): 1257-1272.
§ Chua, Leon O. and Roska, Tamas (1993). The CNN paradigm. IEEE Transactions on Circuits
and Systems 40(3): 147-156.
§ Chua, Leon O. (2006, 2007, 2009). A nonlinear dynamics perspective of Wolfram's new kind
of science (Volumes I, II, III). World Scientific, Singapore .
§ Ercsey-Ravasz, Maria-Magdolna; Néda, Zoltan and Roska, Tamas (2008). Statistical Physics
on Cellular Neural Network Computers. Physica D: Nonlinear Phenomena 237(9): 2051-2068.
§ Gilli, Marco; Roska, Tamas; Chua, Leon O. and Civalleri, Pier Paolo (2002). CNN dynamics
represents a broader class than PDEs. International Journal of Bifurcation and Chaos12(10):
2051-2068.
§ Kocsárdi, Sándor; Nagy, Zoltán; Csík, Árpád and Szolgay, Péter (2009). Two-dimensional
Compressible Flow Simulation on Emulated Digital CNN-UM. International Journal of Circuit
Theory and Applications 37(4): 569-585.
§ Roska, Tamas and Chua, Leon O. (1993). The CNN universal machine: an analogic array
computer. IEEE Transactions on Circuits and Systems II 40(3): 163-173.
§ Roska, Tamás (2003). Computational and computer complexity of analogic cellular wave
computers Journal of Circuits, Systems, and Computers 12(4): 539–562.

Internal references

§ Valentino Braitenberg (2007) Brain. Scholarpedia, 2(11):2918.

§ Olaf Sporns (2007) Complexity. Scholarpedia, 2(10):1623.

§ James Meiss (2007) Dynamical systems. Scholarpedia, 2(2):1629.

§ Rodolfo Llinas (2008) Neuron. Scholarpedia, 3(8):1490.

§ Dale Purves (2009) Neuroscience. Scholarpedia, 4(8):7204.

§ Gregoire Nicolis and Anne De Wit (2007) Reaction-diffusion systems. Scholarpedia,

2(9):1475.

§ Philip Holmes and Eric T. Shea-Brown (2006) Stability. Scholarpedia, 1(10):1838.

Recommended readings

§ Chua, Leon O. and Roska, Tamas (2002). Cellular Neural Networks and Visual Computing.
Cambridge University Press, Cambridge, UK. ISBN 978-0521652476
§ Chua, Leon O. (1998). CNN: A Paradigm for Complexity. World Scientific, Singapore . ISBN
978-0521652476
§ Kek, Laszlo; Karacs, Kristof and Roska, Tamas (2009). Cellular wave computing library,
version 2.1 (templates, algorithms and programs). Available: http://cnn-
technology.itk.ppke.hu/Library_v2.1b.pdf SZTAKI, Hungarian Academy of Sciences,
Budapest.
§ Roska, Tamas and Rodriguez-Vasquez, Angel (Eds.) (2001). Towards the Visual
Microprocessor. Wiley , New York. ISBN 978-0471956068
§ Roska, Tamás (2007). Circuits, computers, and beyond boolean logic. International Journal of
Circuit Theory and Applications 35(5-6): 485–496.
§ Cao, Jinde (2001). Global stability conditions for delayed CNNs. IEEE Transactions on
Circuits and Systems I 48(11): 1330-1333.
§ http://www.anafocus.com: Vision systems based on the CNN technology
§ http://www.eutecus.com: Vision systems based on the CNN technology