(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 6, June 2011
Evolving Fuzzy Classification Systems fromNumerical Data
Pardeep Sandhu
Department of Electronics& Communication,Maharishi MarkandeshwarUniversity, Mullana,Haryana, INDIAer.pardeepsandhu@gmail.com
Shakti Kumar
Computational IntelligenceLaboratoryInstitute of Science andTechnology, Klawad,Haryana, INDIAshaktik@gmail.com
Himanshu Sharma
Department of Electronics& Communication,Maharishi MarkandeshwarUniversity, Mullana,Haryana, INDIAhimanshu.zte@gmail.com
Parvinder Bhalla
Computational IntelligenceLaboratoryInstitute of Science andTechnology, Klawad,Haryana, INDIAparvinderbhalla@gmail.com
Abstract
—
Fuzzy Classifiers are an important class of fuzzysystems. Evolving fuzzy classifiers from numerical data hasassumed lot of significance in the recent past. This paperproposes a method of evolving fuzzy classifiers using a threestep approach. In the first step, we applied a modified FuzzyC
–
Means Clustering technique to generate membershipfunctions. In the second step, we generated rule base usingWang and Mendel algorithm. The third step was used toreduce the size of the generated rule base. This way ruleexplosion issue was successfully tackled. The proposedmethod was implemented using MATLAB. The approachwas tested on four very well known multi dimensionalclassification data sets. The bench mark classification datasets contain: Iris Data, Wine Data, Glass Data and PimaIndian Diabetes Data sets. The performance of the proposedmethod was very encouraging. We further implemented ouralgorithm on a Mamdani type control model for a quickfuzzy battery charger data set. This integrated approach wasable to evolve model quickly.
Keywords
—
Linguistic rules, Fuzzy classifier, Fuzzy logic,Rule base.
I.
I
NTRODUCTION
The theory of fuzzy sets and fuzzy logic was introducedby Lotfi A. Zadeh through his seminal paper in 1965 [1].Both these, fuzzy set theory and fuzzy logic act as apowerful methodology for dealing with imprecision andnonlinearity in an efficient way [2], [3]. As far as the needof fuzzy set theory is concerned, there are numerous
situations in which classical set theory of 0‘s and 1‘s is not
sufficient to describe human reasoning. Thus, for suchsituations we need a more appropriate theory that can also
define membership grades in between ‗0‘ and ‗1‘ thereby
providing better results in terms of human reasoning.Fuzzy set theory attempts to do this.Further this theory of fuzzy logic leads to thedevelopment of fuzzy logic based systems, the systemswhich are capable of making a decision on the basis of knowledge or intelligence provided to the system throughlinguistic rule bases. As a particular combination of inputis given to the system, system on the basis of knowledgeembedded into it in the form of linguistic rules makes adecision and processes those inputs. As the intelligence of these systems depends upon linguistic rule base, thesesystems are also called as Fuzzy Rule Based Systems(FRBSs) [4]. These systems have been successfullyapplied to a wide range of problems from different areaspresenting uncertainty and vagueness in different ways
[5], [6], [7]. These FRBS‘s can be categorize
d asknowledge based systems and data driven systems. Thereare two ways of providing knowledge to the systems. Infirst type of systems called knowledge driven modeling,the rule base is provided by an expert who has thecomplete knowledge of the domain while in second typeof models called data driven models, this rule base isgenerated from available numerical data [8].In data driven systems to automatically generate therule base, a number of classical approaches like Hong and
Lee‘s Algorithm [9], Wang
and Mendel Algorithm [4],[6], [10], [11], [12], Online Learning Algorithm [13],Multiphase Clustering Approach [14] and soft computingtechniques like Artificial Neural Networks [15], [16], [17],Genetic Algorithm [18], [19], Swarm Intelligence basedtechniques [20], Ant Colony Optimization [21], ParticleSwarm Optimization [22], Biogeography basedOptimization [23], Big Bang
–
Big Crunch Optimizationtechnique [24] are available in the literature [25].This paper is based on an integrated approach thatmakes use of a modified Fuzzy C
–
Means Clusteringapproach (FCM) [26] and Wang and Mendel method [6].The approach was implemented in MATLAB for fuzzyclassification problems [27] of Iris data of Fisher [28],Wine data, Glass data, Pima Indian Diabetes (PID) dataand Battery Charger data (control problem) [29]. A system
was evolved using set of training examples and system‘s
performance was then evaluated using test data set for thegiven system. The system performances were evaluated interms of Average Classification Rate (for classificationproblems) and Mean Square Error (for control problem).The paper is organized as follows: Section II introducesFuzzy Logic Based Systems. Section III discusses theproposed integrated approach and WM method for rulebase generation. In section IV the result analysis alongwith the comparative study for above mentioned standarddata sets are shown and section V includes conclusions.
139http://sites.google.com/site/ijcsis/ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 6, June 2011
II.
F
UZZY
R
ULE BASED
S
YSTEMS
Fuzzy logic is a mathematical approach to emulate thehuman way of thinking and learning [30]. This logic is anextension of classical set theory which says a fuzzy set is aclass of objects with a continuum of grades of membership. Such a set is characterized by a membershipmapping the elements of a domain, space or universe of
discourse ‗U‘ to the interval {0, 1}. If ‗U‘ is a collectionof objects denoted by x, then a fuzzy set ‗A‘ in theuniverse of discourse ‗U‘ can be defined as a set of
ordered pairs as shown in equation (1) [5], [8]:
A x x A x A
ii
)(,
(1)Here x refers to i
th
element of the set and µ
A
(x
i
) is themembership grade of x
i
i
n set ‗A‘.
Fuzzy Logic Based Systems or Fuzzy Rule BasedSystems (FRBS) are intelligent systems
those are based onmapping of input spaces to output spaces where the way of representing this mapping is known as fuzzy linguistic rules.These intelligent systems provide a framework for representingand processing information in a way that resembles humancommunication and reasoning process.
Figure 1. Fuzzy Logic System
Each fuzzy rule based system, typically possesses afuzzy inference system (shown in Figure 1) composed of four major modules: Fuzzification module, InferenceEngine, Knowledge Base and Defuzzification module[31]. The fuzzification module performs thetransformation of crisp inputs into fuzzy domain values. Itis mainly done to find the belongingness of data sets todifferent membership functions. The fuzzification can beperformed by either with the help of domain experts ordirectly from the available numerical data. These fuzzydomain values are then processed by inference enginewhich is composed of composition, implication andaggregation processes. The method of processing theinputs is supplied by the knowledge base and rule basemodule as it contains the knowledge of the applicationdomain and the procedural knowledge. Finally, theprocessed output of inference engine is transformed fromfuzzy domain to crisp domain by defuzzification module.One of the biggest challenges in the field of modelingfuzzy rule based systems is the designing of rule base as itis characterized by a set of IF
–
THEN linguistic rules. Thisrule base can be defined either by an expert or can beextracted from numerical data using any computerizedtechniques as mentioned in section I. A rule in fuzzydomain can be represented by equation (2):
Rule: IF antecedent……THEN consequent…….
(2)The antecedent part provides the input variableconditions using IF statements and consequent providesthe output using THEN statements. For example, if X andY are the input and output universes of discourse of a
fuzzy system with a rule base of size ‗N‘, then the rule
will be of the form as shown by equation (3):Rule i
th
: IF x is A
i
THEN y is B
i
(3)Where, x and y represent input and output fuzzylinguistic variables respectively, and A
i
Є X and B
i
Є Y(1≤ i ≤N) are fuzzy sets representing linguistic values of x
and y [5].In Mamdani type systems the consequent is representedusing fuzzy sets while in Sugeno type systems, it is afuzzy singleton. Also in TSK type systems, it is a functionof inputs [23].III.
P
ROPOSED
A
PPROACH
We first broke the system identification problem intothree sub
–
problems and solved these one by one asfollows:1.
Classify all the relevant input and output domainsinto various membership functions using modifiedFCM method [26].2.
Apply Wang and Mendel
algorithm [6] for creatinga fuzzy rule base, evolved as a combination of rulesgenerated from numerical examples and linguisticrules supplied by human experts.3.
Keep the number of rules to bare minimum. Weused a rule reduction technique as proposed in [32],[33] to keep the rule base as compact as possible.The backbone of this approach is the Wang and Mendel
algorithm [6] which has proved to be very effective.Suppose the given set of desired input
–
output data pairsis:
,.......;,,;,
)2()2(
2)2(1)1()1(
2)1(1
y x x y x x
(4)Here x
1
, x
2
are inputs and y is the output. The problemformulation consists of generating fuzzy rules and to usethese rules to determine a mapping from inputs (x
1
, x
2
) tooutput (y).The following steps present our integrated approach:
Step 1: Divide the input output spaces into fuzzyregions:
We divide input spaces into desired number of membership functions using modified FCM [26].Assuming that the domain intervals of inputs x
1
, x
2
andoutput y (equation (4)) lies in [x
1-
, x
1+
], [x
2-
, x
2+
] and[y
-
, y
+
]. Here, the domain interval means the values for aparticular variable will lie in this interval. Each of theseinput and output, spaces are partitioned into (2N+1)regions. The number N can be different for each of thevariables. E.g. if the value of N = 2, then there will be five
140http://sites.google.com/site/ijcsis/ISSN 1947-5500
(IJCSIS) International Journal of Computer Science and Information Security,Vol. 9, No. 6, June 2011
membership functions [6]. A number of other methodsare also available to divide the input output spaces intofuzzy regions.
Step 2: Generate fuzzy rules from given input
–
output data pairs:
In this step, first the degree of a given data set (x
1(i)
,x
2(i)
; y
(i)
) into different fuzzy membership functions aredetermined.Second, assign a given data set (x
1(i)
, x
2(i)
; y
(i)
) to theregion with maximum degree and obtain one rule fromone data set.
Step 3: Assign a degree to each rule:
A degree to each generated rule can be assigned usingfollowing formula of equation (5):
)()()(
21
y x xrule D
C B A
(5)That is the product of membership grade of input x
1
in
fuzzy set ‗A‘, membership grade of
input x
2
in fuzzy set
‗B‘ and membership grade of
output
y in fuzzy set ‗C‘.
Also at this point if an expert is available and he assignshis degree of belief in the correctness of a particular data
set then that degree ‗m‘ must be multiplied with
the aboveexpression.
Step 4: Create a combined fuzzy rule base:
The combined fuzzy rule base is assigned rules fromeither those generated from numerical data or linguisticrules (we assume that a linguistic rule also has a degreethat is assigned by the human experts and reflects the
expert‘s belief of the importance of the rule). Also, if there
is more than one rule having same antecedents butdifferent or same consequents then rule with maximumdegree is to be selected. In this way, both numerical andlinguistic information are represented by a commonframework
–
the combined fuzzy rule base.
Step 5: Determine a mapping based on the combined fuzzy rule base:
Defuzzification strategy is used to determine the outputcontrol for given inputs. This step performs nothing butthe same operation as defuzzification module performs ina fuzzy inference system.
Step 6: Rule reduction:
This step is used to reduce the number of redundantrules from the rule base. Thus the main objective of thisstep has been to deal with rule explosion issue which if left untackled may lead to a rule base with unmanageable,large number of rules in the rule base.This procedure can easily be extended to general multi
–
input multi
–
output cases. So, the approach can be viewedas a very general
‗model–free trainable fuzzy system‘ for
a wide range of applications, where model free means nomathematical model is required for the problem andtrainable means the system learns from examples andexpert rules, and can adaptively change the mapping whennew examples and expert rules are available.IV.
R
ESULT
A
NALYSIS
This section presents the performances obtained by ourintegrated approach that uses modified Fuzzy C
–
MeansClustering [26] and Wang and Mendel algorithm [6] toevolve fuzzy rule based systems. We applied our approachon four very well known classification data sets frommachine learning repository and one control data set. Ineach experiment, the input and output domain intervals arefuzzified using modified FCM approach. The training datasamples are selected from available data sets incorrespondence with the peaks of the input membershipfunctions. This sequence is used to train the systemswhich are then tested using testing data sets.
A.
Example 1: Iris Data Classification Problem
The proposed approach has been applied on Iris Dataclassification problem. The Iris data set is a widely usedbenchmark for classification and pattern recognitionstudies [27], [28]. The dataset contains 150 samples of data (50 samples for each species) with four attributes asinputs, Sepal Length, Sepal Width, Petal Length and PetalWidth and three classes of iris plants namely: Iris Setosa,Iris Versicolor and Iris Virginica as output. All the inputvariables have measurement units in centimeter while theoutput is the type of iris plant. The learning sequenceincludes 24 data samples while the system is tested on all150 data samples. By applying the proposed method onthe learning sequence, a set of 24 classification rules (onerule per training data sample) is obtained. From thiscombined rule base, the redundant rules are then removedusing rule reduction algorithm [32], [33] and the final rulebase composing 4 rules are shown in Table I.
TABLE I. C
LASSIFICATION
R
ULE
B
ASE
F
OR
I
RIS
D
ATA
C
LASSIFIER
SepalLengthSepalWidthPetalLengthPetalWidthClass
SL
–
L
SW
–
M PL
–
L PW
–
L SetosaSL
–
M
SW
–
L PL
–
M PW
–
M VersicolorSL
–
M
SW
–
L PL
–
H PW
–
M VirginicaSL
–
M
SW
–
L PL
–
H PW
–
H Virginica
Here, L
–
Low, M
–
Medium, H
–
High
TABLE II. C
LASSIFICATION
R
ATES
F
OR
I
RIS
D
ATA
C
LASSIFIER
(P
ROPOSED
A
PPROACH
)
Numberof RulesSetosa Versicolor VirginicaAverageRate
4
98.00% 100.00% 94.00%
97.33%
3
98.00% 100.00% 90.00%
96.00%
Table II shows the class wise classification rates alongwith the effect of variations in the size of the rule base.Table III presents a comparative analysis of differentalgorithms with the proposed integrated approach for Iris
141http://sites.google.com/site/ijcsis/ISSN 1947-5500
Search History:
Result 00 of 00
00 results for result for
p.
Evolving Fuzzy Classification Systems from Numerical Data
Fuzzy Classifiers are an important class of fuzzy systems. Evolving fuzzy classifiers from numerical data has assumed lot of significance in the recent past. This paper proposes a method of evolving fuzzy classifiers using a three…
(More)
436
Reads
1
Readcasts
0
Embed Views
Recommended
More From This User
Notes
Load more
