(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 5, August 2010
A New Learning Method for Cellular Neural NetworksTemplates based on Hybrid of Rough Sets and GeneticAlgorithms
Elsayed Radwan
*
, Omaima Nomir
Department of Computer Science, Faculty of CIS,
Mansoura University, Egypt
E-mails:
elsfadwan@yahoo.com,
o.nomir@umiami.edu
Eiichiro Tazaki
Department of Control and Systems Engineering, ToinUniversity of Yokohama, Japan
E-mail:
tazaki@intlab.toin.ac.jp
Abstract
—
A simple method for synthesizing and optimizingCellular Neural Networks is proposed. Based on the RoughSets concept and the comparison principles for ordinarydifferential equations, a mathematical system of inequalitiesand the optimal cloning template structure are discovered.By solving this system of inequalities, the derived parametersare represented to be the Cellular Neural Networkstemplates. These parameters guarantee correct operations of the network. To represent a more robust template, arandomized search and an optimization technique guided bythe principles of evolution and nature genetics withconstrained fitness and, penalty functions, has beenintroduced. Applying our introduced method to differentapplications shows that our new method is robust.
Keywords-component; Rough Sets; Cellular Neural Networks,Comparison principles; Template Robustness; Genetic Algorithms
I.
I
NTRODUCTION
Cellular Neural Networks [2], CNN were invented to circumventthis curse of interconnecting wires. The problem gained by the fullyconnected Hopfield Network [10], is by decreasing that and thereshould be no electrical interconnections beyond a prescribed sphere of influence. This makes it easy to be implemented via physics device asVLSI (Very Large Scale Integrated) Circuit. During the CNNinvention period, due to the lack of any programmable analogic CNNchips, the templates were designed to be operational on ideal CNNstructures. These structures were simulated on digital computers.Later, several templates learning and optimization methods weredeveloped. The goal of these methods was template generation,dealing with ideal CNN behavior but without much regard torobustness issues. As a result, a large number of templates wereintroduced. Some of these templates were designed by using templatelearning algorithms, but most of them were created relaying on ad hocmethods and intuition. Since the programmable CNN chips werefabricated, many of these templates were found to work incorrectly intheir original form (i.e. as used in software simulators). Consequently,new chip- independent robust template’s design methods wereintroduced. According to previous studies [8], the actual templatevalues at each cell will be different from the ideal ones. This is mainlydue to the noise in the electrical components, superfluous cells as wellas the template parameters. This results in some cells respondingerroneously to some inputs. An improvementshould be achieved by designing robust templates for a givenCNN’s operations so that they are most tolerant against parameter deviations. This can be achieved by removing the cells that have noeffect on classifying the output, and superfluous cells, removing noisein the training data, and discovering the optimal template parameters.In this paper, we introduce an analytical method to synthesize a CNNfor solving a given problem. Our introduced method relies on Roughsets concepts [15] in discovering the optimal template structure byremoving the superfluous neighboring cells which have no effect onclassifying the cell’s output. Another important concept of rough setsis its ability to determine the significance of each neighbor cell. Thisrough sets’ feature gives us the idea to define a new measure called thesign measure. This measure is used in deducing the relation among thetemplate parameters. Also, by rough set concepts the similarities in theinput data are discovered and excluded, which will result in reducingthe learning time. Moreover, it is able to discover the optimal localrules of the most simplified construction, which (almost) preserveconsistency with data and classify so far unseen objects with thelowest risk of error. Therefore the capability of classifying moreobjects with high accuracy, increase the CNN template robustness, andthat needs neglecting cells being the source of redundant information.Depending on the local rules, our method uses a simple procedure of the so-called comparison principle [3], which provides bounds on thestate and output waveforms of an analog processing cell circuit. Wewill be able to find conditions on the elements of the CNN, ensuring acorrect functioning of the CNN for a particular application. To find theglobal minima, even in a noisy and discontinuous search space andwithout using differentiable information about the cost function,Genetic Algorithms with constrained fitness function [17] that takesinto account the hardware implementation is used. This research work is an extension of the previous work [6], where a special case of CNNis handled. Rough sets are used in discovering the optimal CNNtemplate structure. Also, the comparison principle technique is used totreat the regular discovered rough sets' rules to be a set of inequalitiesthat constraints the CNN structure. The problem of uncoupled CNN indesigning a simple application of edge-detection CNN. is solved.The rest of this paper is organized as follows: Section 2 explainsthe role of rough set concepts in reasoning about cells and concludesthe optimal local rules that describe the CNN dynamic. Section 3,describes the Genetic algorithm in learning the cloning templates.Sections 4 presents the experimental results on some simpleapplications and then section 5 concludes the paper.
II.
ROUGH
SETS
IN
R
EASONING ABOUT CELLS
Cellular Neural Networks [4] is any spatial arrangement of nonlinear analogue dynamic processors called cells. Each cellinteracts directly within finite local neighbors that belong to thesphere of influence
}|)||,max(||{)(
r l jk icij N
klr
≤−−=
and characterized by a nonlinear dynamical system. This dynamicalsystem has an input
u
, a state
x
evolved by time according to some prescribed dynamical laws, and an output
y
, which is a function of the state. The cell dynamics are functionally determined by a smallset of parameters which control the cell interconnection strengthcalled templates. It is characterized by the following equations [1].
155http://sites.google.com/site/ijcsis/ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 5, August 2010
zt u Bt y At x Rt x
dt d C
ij N klklklij N klklklijijij
r jir
+∑+∑+−=
∈∈−
)(,,1
)()()()(
),(
(1)
|)1)(||1)(|(5.0))(()(
−−+==
t xt xt x f t y
ijijijij
(2) (2)
N j M i
z zt u x
ijij
≤≤≤≤
≤≤≤−≤≤−
1,1,||,1|)(1,1)0(1
max
(3) (3)A and B are the feedback and the feed-forward templatesrespectively, and
z
is the search bias threshold. The machine usesthe simple CNN in a time-multiplexed fashion, analogous to the ALUof the microprocessor, by controlling the template weights and thesource of the data inputs for each operation. The machine suppliesmemory and register transfers at each cell that allow the outputs of the CNN operations to be combined and/or supplied to the inputs of the next operations, thereby allowing more complex algorithms to beimplemented. Then, for any input pattern
U
, the output for each cell
)(
∞
ij
y
is uniquely determined by only a small part of
U
, depictedin Figure 1 where the radius of the sphere of influence
1
=
r
,exposed to
)12()12(
+×+
r r
transparent window centered at cell
ij
C
. According to the complete stability theorem of the uncoupledCNN [1] [5], the output
)(
∞
ij
y
is considered as a functionin
)12)(12(
++
r r
of input variables in addition to a predefinedinitial state
0
x
,
),...,,(
)12)(12(10
++
=
r r ij
uu x f y
. Thefunctionality of the uncoupled CNN is a one-one mapping from
U
to
Y
for a predefined initial state
0
x
that describe the dynamic at
0
=
t
.Hence, the dynamic for space invariant uncoupled CNNcan be completely described by a Knowledge Representation System,
KRS
,
),(
0
Y C X U S
∪∪=
where
U
is the whole universe of input pattern and
C
is the neighbor cells,
Y
is the output from a predefined initial state
0
X
,
C Y
∉
[4]. Then, every row
h
in
S
isconsidered as an if-then rule by the form;
hhhhh
y ythenucucuct xcif
=====
))(&...&)(&...&)(&))(((
99551100
(4)To summarize, it can be described as a
CY
decision rule
ψ φ
→
where the predecessor
φ
is a conjunction of
1)12)(12(
+++
r r
of input cells,
),(
hii
uc
, and the successor
ψ
is the classified output. This means, the whole
KRS
looks like acollection of
CY
decision rules or in short
CY
decisionalgorithm,
U mY C Dec
mk k k
≤≤→=
=
2,}{),(
1
ψ φ
. Thisdecision algorithm can be treated by an algorithm for synthesis of decision rule from decision table. Rough Sets [12][15] provides amathematical technique for discovering the regularities in data, whichaims to obtain the most essential part of the knowledge thatconstitutes the reduced set of the system, called the reduct. It dependson the analysis of limits of discernibility of subsets of objects fromthe universe of discourse
U
. For this reason, it introduces two subsets,the lower and upper approximation sets. With every subset of attribute
C C
⊆
, any equivalence relation
C
I
on
U
can easily beassociated to;
)}()(,everyfor :),{(
yc xcC cU y x I
iiiC
=∈∈=
(5)Then
ii
cC cC
I I
∈
∩=
. If
U X
⊆
, the sets
}][:{
X xU x
C
⊆∈
and
}][:{
φ
≠∩∈
X xU x
C
where
C
x
][
denotes the equivalence class of the object
U x
∈
relative to
C
I
, which are called the
C
-lower
and
C
-upper
approximation of
X
in
S
.
Through this paper, rough set relies on discovering theconsistency relation among the rules, by means of decision language,and determining the dependencies among data. The rules of the mostsimplified construction, (almost) preserve consistency with data, arelikely to classify so far unseen object with the lowest risk of error.Therefore, to be capable of classifying more objects with highaccuracy, we need to neglect cells being the source of redundantinformation, i.e. use the reduct of attributes.
Definition 1
: if for every
ψ φ
→
,
ψ φ
′→′
),(
Y C Dec
∈
, wehave
φ φ
′=
implies
ψ ψ
′=
, then
),(
Y C Dec
is called consistentalgorithm, otherwise it’s called inconsistent algorithm. Also wedefined the positive region of
),(
Y C Dec
denoted
),(
Y C POS
to be the set of all consistent rules in the algorithm.A cell attribute
C c
i
∈
is dispensable (superfluous) in
),(
Y C Dec
if
),(
Y C POS
=
)},{(
Y cC POS
i
−
; otherwise thecell attribute
C c
i
∈
is indispensable in
),(
Y C Dec
. Thealgorithm
),(
Y C Dec
is said to be independent if all
C c
i
∈
areindispensable in
),(
Y C Dec
.The set of cell attributes
C C
⊆
will be called a reduct of
),(
Y C Dec
, if
),(
Y C Dec
is independent and
),(
Y C POS
=
),(
Y C POS
. Based on the significance of each cell,the algorithm to compute the reduct is as follows;1-
Let
ϕ
=
R
,
},,,,{
)12)(12(210
++
=
r r
ccccC
K
and
0
=
i
2-
Compute the accuracy measure of the original table
),(),(
Y C DecY C POSk
=
3-
While
))12)(12((
++<=
r r i
doa- Compute the accuracy measure by dropping the cell
i
C
,
),()},{(
Y C DecY cC POS
k
ii
−=
b- If (
0
=−=
ic
k k
i
γ
)- Let
}{
i
c R R
∪=
and-
}{
i
cC C
−=
c-
1
+=
ii
i
c
γ
is called the cell significance which represents the bifurcationsin the CNN dynamical system caused by removing the cell
i
c
. If
k
156http://sites.google.com/site/ijcsis/ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 8, No. 5, August 2010
equals one, i.e. consistent algorithm, then the algorithm describes acomplete stable dynamic. Suppose that the CNN dynamic could berepresented by a decision algorithm and then study the affection of consistency relation in realizing the optimal template structure of asingle layer CNN through the following theorems.
Theorem 1:
Any consistent algorithm can be recognized by a CNNtemplate for which, for all linear cells, there is no other directconnected linear cells, where a cell
ij
C
is directly connected to a cell
mn
C
if
r mi
≤−
and
r n j
≤−
, i.e.
)(
ij N C
r mn
∈
and thefeed-back synapse
0
,
≠
−−
n jmi
A
.
Proof:
Let
L
C
be a linear cell, i.e.
)()(
t yt x
=
. For any consistentalgorithm, all cells that are directly connected to
L
C
must haveconstant output. Then the dynamics of
)(
t x
L
in the linear regiongoverned by
qat x
dt dx
c L L
+−=
)1)((
where
q
comprises thecontribution of the neighbor output values from the input, bias, and boundary which is constant by assumption as long as
L
C
is linear.
00
Aa
c
=
, where
00
A
is the center element of the
A
-template. Thesolution is a single exponential function with a positive argument,which guarantees that the equilibrium lies in the saturation region.Hence the sign of
dt t dx
L
)(
determines the output values of theneighboring cells and can not change while the linear region.Therefore, the template is uncoupled CNN or there is no directconnected linear cells.
Corollary 1
: Any inconsistent algorithm can not be realized by asingle layer space invariant CNN without directly connected cells.
Proof:
we prove that using contradiction by considering the oppositei.e. consider its inconsistent algorithm and can be represented byCNN with no directly connected cells. Then, the CNN dynamic can be represented by
z BU W I A AW AX
dt dX
+=−=+=
and)1(,
00
,
I
is theidentity matrix (6)Then,
t A
eC X
)1(0
00
−
=
and
),(
00
W AC C
=
is a linear functiondepends on a self-feedback constant value and the offset level whichis a function of the input pattern. Hence the trajectory depends onanential function on time, i.e. it’s a continuous monotonic functionconverges to a single equilibrium point. Thus, consider that
),(
Y C Dec
∈→
ψ φ
and since
φ
based on the input pattern,
0
C
is a constant value, and
ψ
is determined by a linear piecewisefunction in Equation (2) as function in the trajectory which convergesto a single equilibrium point. Hence
ψ
is a one-one function whichcontradicts the definition of inconsistence. Therefore, we reject our assumption which completes the proof.Since consistency of the algorithm gives no promise for thelinearly separable, such as
XOR
logic function which is consistent butnon-linearly separable, then the algorithm should be checked for linear separability. If it is a consistent algorithm and linearlyseparable, it can be recognized by a stable dynamic with memoryless parameters, and synaptic weights, according to the followingtheorem;
Theorem 2:
Any consistent algorithm which is linearly separable andhas
0
c
as a superfluous attribute, it can be recognized by a singlelayer CNN with memoryless synaptic weight.
Proof:
Since CNN is a massively parallel architecture working with analogsignals, and as the path of information is an analog to digitalconverter, then our proof will be concentrated on the binary outputonly.
Case 1: (binary input signals)
Since any consistent algorithm with binary signals can be seen as atruth table, then it can be determined by a statement form in whichthe only connectives occurring are from amongst, (
∨∧
,~,
, negation,conjunction and disjunction functions). Since for any local linearlyseparable Boolean function, there exists a barrier (plane) satisfies thatthe output at each cell
],sgn[
b xa y
−><=
. According to [1],they proved that any local Boolean function
),...,,(
921
x x x
β
of nine variable is realized by every cell of an uncoupled CNN. Thishappens if and only if
(.)
β
can be expressed explicitly by theformula
[ ]
b xa
−><=
,sgn
β
where
><
xa
,
denoted the product between the vectors
],...,,[
921
aaaa
=
and
],...,,[
921
x x x x
=
, where
Rb Ra
i
∈∈
,
and
}1,1{
−∈
i
x
isthe i
th
Boolean variable,
i
=1,…,9. Hence, there exists a single layer CNN with memoryless synaptic weights that realize the output,which satisfy the proof.
Case 2: (analog input signals)
We prove by considering the opposite, i.e. the output can not berecognized by a single layer CNN. Thus, by a single layer there existsan error corresponding to some cells
E
C
. This means that some cellsremains in the linear saturation region or in the opposite saturationregion. From theorem 1, any cell in the CNN including
E
C
should be realized a template for which, for all linear cells, there is no other directly connected linear cells, which is completely stable dynamic,i.e. all cells should belong to only one of the positive or negativesaturation region. Hence,
E
C
should be in opposite saturationregion, this case should be happened when
E
C
located in one of thedegenerate cases. From the assumption about the binary output, thereis only one degenerate case when the self-feed back
00
A
is greatestthan one (i.e. CNN dynamic depends on the initial state), whichcontradicted with theorem 1 and
0
c
is the superfluous attribute. Thisleads us to reject our assumption and the consistent algorithm isrecognized by a single layer. Since the algorithm is linearlyseparable, the output can be recognized by a single layer CNN withmemoryless synaptic weights.After determining the set of reduct
C
and cells significance, weconstruct the CNN structure by removing the cells that correspondingto the attributes in the set
R
, the set of superfluous cells. Also, thedecision table should be modified by removing the columnscorresponding to the superfluous cells. Coupled to this, if
0
c
belongs
157http://sites.google.com/site/ijcsis/ISSN 1947-5500
Search History:
Result 00 of 00
00 results for result for
p.
A New Learning Method for Cellular Neural Networks Templates based on Hybrid of Rough Sets and Genetic Algorithms
A simple method for synthesizing and optimizing Cellular Neural Networks is proposed. Based on the Rough Sets concept and the comparison principles for ordinary differential equations, a mathematical system of inequalities and the optimal cloning template…
(More)
205
Reads
1
Readcasts
0
Embed Views
More From This User
Notes
Load more
