A project report submitted in partial fulfillment of the requirements for the award of the degree of Master of Engineering (Electrical – Mechatronics and Automatic Control)

Faculty of Electrical Engineering Universiti Teknologi Malaysia




To my beloved mother and late father, My kind brothers and sisters And not forgetting to all my friends For their Love, Sacrifice, Encouragement, and Best Prayers



Praise be to Allah S.W.T to Whom we seek help and guidance and under His benevolence we exist and without His help this project could not have been accomplished.

I would like to express my sincere thanks and appreciation to Dr. Hj. Mohd. Fauzi bin Othman for his supervision, advice, guidance, encouragement and insightful comments given throughout the course of completing this project.

I am also indebted to the Universiti Teknologi Malaysia for their encouragement and financial support during my study. Not forgetting to all my fellow postgraduate students and friends for their moral support and given throughout the entire masters programme. Without their continued support and interest, this project would not have been realized.

Last but not least, my gratitude also goes to all my family members for their continuous encouragement and support. Thank you all.

in this project the neuro fuzzy system or ANFIS was applied for classification and detection purposes.v ABSTRACT The conventional method in medicine for brain MR images classification and tumor detection is by human inspection. The use of artificial intelligent techniques. neural networks. Operator-assisted classification methods are impractical for large amounts of data and are also non-reproducible. . fuzzy logic. The performance of the ANFIS classifier was evaluated in terms of training performance and classification accuracies and the results confirmed that the proposed ANFIS classifier has potential in detecting the tumors. Hence. Decision making was performed in two stages: feature extraction using the principal component analysis (PCA) and the ANFIS trained with the backpropagation gradient descent method in combination with the least squares method. neuro fuzzy have shown great potential in this field. for instance. MR images also always contain a noise caused by operator performance which can lead to serious inaccuracies classification.

. Prestasi pengkelas ANFIS berkenaan dengan prestasi latihan dan ketepatan pengkelasan telah diuji dan hasilnya telah menunjukkan bahawa pengkelas ANFIS yang dicadangkan mempunyai potensi untuk mengesan barah. Oleh yang ABSTRAK Teknik konvensional di dalam perubatan untuk pengkelasan dan pengesanan barah untuk imej magnetik resonan otak adalah dilakukan secara pemeriksaan manual. Imej magnetic resonan juga lazimnya mengandungi hingar yang berpunca daripada mutu kerja operator yang mana boleh menjurus kepada ketidaktepatan pengkelasan yang serius. Pengambilan keputusan telah dilakukan secara dua peringkat: pengekstrakan sifat menggunakan prinsip analisis komponen dan ANFIS terlatih dengan penggabungan teknik ‘backpropagation gradient descent’ dan juga ‘least squares’. Penggunaan teknik kecerdikan buatan contohnya. neural network. Teknik pengkelasan yang dibantu oleh operator adalah tidak praktikal untuk jumlah data yang besar dan juga tidak dapat dihasilkan. fuzzy logic dan juga neuro fuzzy telah menunjukkan potensi yang besar di dalam bidang ini. di dalam projek ini sistem neuro fuzzy atau juga dikenali sebagai ANFIS telah diaplikasikan untuk tujuan pengkelasan dan pengesanan.

1 2.1 2.5 1.2.1 1.3 1.6 Project Background Magnetic Resonance Imaging Problem Statement Objectives Scope of Work and Methodology Thesis Outline 1 1 2 5 5 6 7 2 LITERATURE REVIEW 2.2 Related Works Artificial Neural Networks 2.2 1.2.2 Learning from Biology Generic Model of Neural Network 8 8 11 11 13 .4 1.vii TABLE OF CONTENTS CHAPTER TITLE PAGE DECLARATION DEDICATION ACKNOWLEDGMENTS ABSTRACT ABSTRAK TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES LIST OF ABBREVIATIONS LIST OF SYMBOLS LIST OF APPENDICES ii iii iv v vi vii x xi xiii xiv xv 1 INTRODUCTION 1.

7 3.8.6 Methodology Proposed Method Transformation of Image to Data Form Pixels Labeling Region of Interest Extraction Feature Extraction using Principal Component Analysis (PCA) 3.3 3.5 3.5 2.2 3.1 2.3 Bias and Threshold Activation Functions Network Learning Phase 14 15 15 16 17 18 19 20 21 21 Fuzzy Logic Systems 2.6 Data Management and Preparation Data Mining Input Phase FIS Phase ANFIS modeling phase Output phase 27 27 27 28 30 31 32 34 37 37 38 38 39 40 42 .4 2.8.1 Principal Component Analysis (PCA) 3 RESEARCH METHODOLOGY 3.6 Advantages of Fuzzy Logic Disadvantages of Fuzzy Logic Fuzzy Sets Membership functions Components of a fuzzy system Fuzzy Logic Graphical User Interface (GUI) 24 25 2.4 2.4 3.2 2.3 2.6 2.5 3.2.2 3.8 Data Modeling System developing 2.8.3 3.1 3.

3 4.2 5.1 4.2.10 ANFIS Architecture Learning algorithm of ANFIS ANFIS Classifier 43 43 44 45 49 51 52 53 54 55 56 57 58 ANFIS editor GUI FIS Editor Membership Functions Editor Rule Editor Rule Viewer ANFIS Model Structure Modeling Data through ANFIS Model Validation (Checking and Testing Data) 60 5 RESULTS AND DISCUSSIONS 5.4 4.2 Conclusions Future Works 72 72 74 REFERENCES Appendices A-K 77 80 .ix 4 NEURO FUZZY MODELING AND SOFTWARE IMPLEMENTATION 4.6 4.2.3 4.1 5.1 6.3 5.2 4.2.8 4.1 4.99 .7 4.9 4.5 4.4 Result Performance Analysis Noise Consideration Comparisons of Result 62 62 64 66 70 6 CONCLUSIONS AND FUTURE WORKS 6.2 Neuro Fuzzy Model Adaptive neuro fuzzy inference system (ANFIS) 4.

1 5.6 5. 4.5 5.x LIST OF TABLES TABLE NO.8 5.3 5.7 5.4 5.2 5.9 TITLE Two passes in the hybrid learning procedure for ANFIS The ANFIS errors The ANFIS test results The values of statistical parameters Testing data set with 1% noise Testing data set with 2% noise Testing data set with 3% noise The ANFIS test results with noise consideration PNN target results Comparison between PNN and ANFIS PAGE 51 63 66 66 67 67 68 70 70 71 .1 5.

4 2.7 3.1 MRI machine TITLE PAGE 3 4 4 13 14 16 21 21 22 24 28 30 30 31 32 34 36 37 39 40 41 41 42 46 Schematic diagram of MRI machine Brain MRI Biology neuron Generic model of artificial neural network Feedforward Network Membership functions of a crisp set and a fuzzy set Membership grades of a fuzzy set Components of a fuzzy logic system Fuzzy Logic Graphical User Interface (GUI) The proposed system Convert brain image to binary image Pixel coordinate system Advanced coordinate system for matrix 16 by 16 Location of tumor Schematic diagram of a MR image recognizer Training Data Set Basic flow diagram of computations in ANFIS Partial of input phase FIS generated thru genfis1 Information of the ANFIS model ANFIS training errors and total time training Sample of classification results A two-input first-order Sugeno fuzzy model with two rules .xi LIST OF FIGURES FIGURE NO.3 3.1 2.7 3.1 3.11 3. 1.2 1.8 3.10 3.13 4.12 3.5 3.3 2.1 1.4 3.2 2.5 2.3 2.6 2.2 3.6 3.9 3.

8 4.5 5.12 5.6 4.4 5.2 ANFIS architecture.xii 4.11 4.7 4.3 4.5 4.10 4.2 5.6 5.1 6.4 4.9 4.3 5.8 6.7 5.1 5. Two-input first-order Sugeno fuzzy model with nine rules The input space that are partitioned into nine fuzzy regions ANFIS editor GUI FIS editor Membership function editor Rule editor Rule viewer ANFIS model structure Training error Output of ANFIS Training and checking data ANFIS errors The misclassified output of ANFIS classifier a)Membership function before training b)Membership function after training The misclassified output of ANFIS classifier with 1% noise The misclassified output of ANFIS classifier with 2% noise The misclassified output of ANFIS classifier with 3% noise Chart comparison Different slice of brain medical image Block Diagram of the neuro fuzzy approach 46 49 49 53 54 55 56 57 58 59 60 62 63 64 65 68 69 69 71 75 76 .2 4.

xiii LIST OF ABBREVIATIONS AFLC ANFIS ANN CAT COG COS CT FCM FIS GUI MF MLP MR MRI NMR NN PCA PET PNN RF RGB ROI - Adaptive Fuzzy Leader Clustering Adaptive Neuro Fuzzy Inference System Artificial Neural Network Computed Axial Tomography Centre of Gravity Centre of Singleton Computed Tomography Fuzzy C-Means Fuzzy Inference System Graphical User Interface Membership Function Multi Layer Perceptron Magnetic Resonance Magnetic Resonance Image Nuclear Magnetic Resonance Neural Network Principal Component Analysis Positron Emission Tomography Probabilistic Neural Network Radio Frequency Red Green Blue Region of Interest .

xiv LIST OF SYMBOLS θ i w b O ∑ Π x e E ∈ µA(x) ∩ ∪ - threshold Input signals Weight the neurons Bias signal Output signals Summation of all incoming signals Product of all the incoming signals Universal sets error total error Subset Membership functions of A Intersection @ AND logic operation Complement @ OR logic operation activation function Rows x columns training image of image i pixel vector feature vectors f MxN i Φ ωi .

xv LIST OF APPENDICES APPENDIX A B C D E F G H I J K TITLE MATLAB source code for transformation image to data PCA Algorithm code CreateDatabase function code EigenMRICore function code TumorRecognition function code Neuro Fuzzy Algorithm code Brain MR Image for Class 1 Brain MR Image for Class 2 Brain MR Image for Class 3 Brain MR Image for Class 4 Brain MR Image for Class 5 PAGE 80 83 85 87 89 91 95 96 97 98 99 .

computer assistance is highly sought in medical institutions due to the fact that it could improve the results of humans in such a domain where the false negative cases must be at a very low rate. Different approaches that can produce medical images must be studied. But the cost incurred in double reading is very high. such as scans. therefore good software to assist humans in medical institutions is of great interest nowadays. With all this effort done in the research field. The idea is to reduce human error as much as possible by assisting . Why is this domain in continues expansion and without and good accepted method? This is due to the fact that in such an important domain. MRI images and other radiological techniques. It has been proven that double reading of medical images could lead to better tumor detection.CHAPTER 1 INTRODUCTION 1. The problem is that it is not very easy to obtain such results. A lot of methods have been proposed in the literature for CT (Computed Tomography). Also. different types of X-rays. the accuracy must be very high and the false negative rate must be low. Also.1 Project Background Automated classification and detection of tumors in different medical images demands high accuracy since it deals with human life. there is still a lot of place for improvements and the medical image processing is a domain in continuous expansion. the technique that produces those images is very important in order to know what to apply to a certain medical image in order to get better results.

In this project the automated classification of brain magnetic resonance images by using some prior knowledge like pixel intensity and some anatomical features is proposed. Neuro fuzzy systems are fuzzy systems which use ANNs theory in order to determine their properties (fuzzy sets and fuzzy rules) by processing data samples. classification or clustering techniques is essential to the developments of neuro fuzzy systems particularly in medical-related problems. Successful implementations of ANFIS in biomedical engineering have been reported. is primarily a tomography imaging technique . fully understanding the detection. MRI. The application of neuro fuzzy systems in the classification and detection of data for MR images problems are not fully utilized yet. MRI is a type of nuclear magnetic resonance (NMR). Furthermore. A specific approach in neuro fuzzy development is the adaptive neuro fuzzy inference system (ANFIS). These include the clustering and classification techniques especially for MR images problems with huge scale of data which consumes time and energy if done manually. including the brain. fuzzy set theory plays an important role in dealing with uncertainty when making decisions in medical applications [1].2 physicians and radiologists with some software that could lead to better results. like other imaging techniques such as CAT and PET. Since currently there are no widely accepted methods. which has shown significant results in modelling nonlinear functions.2 Magnetic Resonance Imaging Magnetic resonance imaging (MRI) is a procedure used to produce high resolution images of the inside of the body. 1. The technique is called magnetic resonance imaging rather than nuclear magnetic resonance imaging because of the public's negative associations with the word nuclear in the late 1970's. This is important since it involve saving human lives. which is used by chemists to study the properties of molecules. therefore automatic and reliable methods for tumor detection are of great need and interest. for classification [3] and data analysis [4]. The ANFIS learns features in the data set and adjusts the system parameters according to a given error criterion [2]. Thus.

This form of imaging is.1 shows a picture of an MRI machine and Figure 1. in some respects. unlike a post-mortem slice. the MRI is formed by an MRI camera or scanner and the slices are composed of groups of picture elements or pixels displayed on a computer monitor.1: MRI machine . In MRI the subject or patient is placed inside the scanner which is made up of a moveable bed-like structure and a large hollow tube. This means that it produces an image in a thin slice through the body. an MRI image may be indistinguishable from a slice taken from a post-mortem body. transformed and then displayed [5].3 ("tomo" means slice). The latter are the raw data of an MRI which are recorded by the computer. to the untrained eye. In fact. This very strong magnetic field is then used to align the hydrogen nuclei of the tissue to be imaged. In order to generate a magnetic field of this strength the coil is cooled to near absolute zero with liquid helium. Figure 1. However. Figure 1. The tube contains a thick coil of wire that generates a very intense magnetic field (between 1.2 shows the schematic of an MRI machine. very much like a real anatomical slice through the body. The RF coil is used to both change the energy state of the hydrogen nuclei and to record the RF output of these perturbations.5 and 4 Tesla) which is strong enough to accelerate a paper clip to near lethal velocities across the distance of a small room. Also placed near the tissue to be imaged is another coil of wire called an RF coil (RF because it generates/reads radio frequency fields).

2: Schematic diagram of MRI machine In MRI.3: Brain MRI .3. water molecules give off radio signals which are converted into high resolution images that look like a picture shown in figure 1. Figure 1.4 Figure 1.

classification and detection of algorithms which have direct applications in many medical problems have become much easier to be implemented and diagnosed. diagnosis with wrong result and delay in delivery of a correct diagnosis decision could occur due to the complexity and cognitive process of which it is involved.4 Objectives The main objective of this project is to propose an efficient classification technique for magnetic resonance (MR) images using neuro fuzzy. Besides. this project tries to find out how the neuro fuzzy system or the hybrid combination of neural network and fuzzy logic techniques can be applied in the classification and detection purposes. The MR images data is by nature. a huge. 1.5 1.3 Problem Statement The conventional method in medicine for brain MR images classification and tumor detection is human inspection. The use of artificial intelligent techniques for instance. Hence. MR images also always contain a noise caused by operator performance which can lead to serious inaccuracies classification. a sample medical image will be tested on the model to observe the classification and detection performance and subsequently optimization . In some extreme scenario. Hence. the pattern matching. neural networks. the study on the fundamental theory of neuro fuzzy system have to be done. With the involvement of soft computing. particularly on the medical field. Operator-assisted classification methods are impractical for large amounts of data and are also non-reproducible. The neuro fuzzy models hereby refer to the ANFIS (Adaptive Neuro Fuzzy Inference System). This includes the neuro fuzzy architecture and the adaptation algorithm for the neuro fuzzy structure itself. complex and cognitive process. fuzzy logic. neuro fuzzy has shown great potential in this field. Accurate diagnosis of MR images data is not an easy task and is always time consuming.

1. Extract the region of abnormal tissues using image processing tool in MATLAB. iii. Develop the neuro fuzzy algorithm for classification using MATLAB. As the ultimate goal of this project. iv. viii. vi. Involves the neuro fuzzy model coding. v. iv. Analyzing the neuro fuzzy model performance accuracy. Deal only with the same size of the brain medical images slice. Distinguish between normal and abnormal of the brain medical images. Training. hopefully an automated system that provides high accuracy of brain medical images classification and symptoms detection that probably can replace the current human visual inspection methods is achieved. ix. ii. testing and tuning or optimization parameters.5 Scope of Work and Methodology The scope of the project is as follows:i. The brain medical images also have to be pre-processed using PCA to reduce the dimensionality of the data.6 of the chosen technique will be carried out. Process the brain medical images using PCA. iii. ii. This project starts with converting the brain medical images into a form of data using MATLAB. simulation. Classify the abnormal brain medical images based on the region of abnormal tissues using neuro fuzzy. The neuro fuzzy model considered hereby refers to ANFIS (Adaptive Neuro Fuzzy Inference System). . adaptation and training of the neuron using the MATLAB. The methodologies of project are as follows: i. Convert the brain medical images into a form of data using MATLAB. vii.

1. In Chapter 1. optimize the performance of the proposed technique. problem statement. Lastly. The neuro fuzzy architecture and software development and implementation for classification are discussed in Chapter 4. this proposed technique will be coded in MATLAB and subsequently to be trained and tested with the brain medical images data. scope of work and methodology are presented. transformation of image into a form of data and also data and ANFIS modelling. . objectives. the conclusion of this project and some recommendations for future work are discussed in Chapter 6. Chapter 3 contains a research methodology of the proposed method. Chapter 5 presents the result from the simulation and discusses the performance analysis. Then. vi. related works and also theories regarding this project are included in Chapter 2. the extraction of a region of interest. Some literature reviews.6 Thesis Outline This thesis is organized as follows.7 v. some overview including the background. Finally.

where it uses neural network’s ability to learn.CHAPTER 2 LITERATURE REVIEW 2. the Fuzzy Neural . respectively. A training set is used to create and train the classifier system. N. (2006) has proposed an approach for detection and specification of anomalies present in medical images. The method is based on the spatial properties of the image features and makes use of multi-scaled representations of the image. After applying the growing region algorithm to extract regions. Each pixel is assigned a degree of membership for each of the three fuzzy subsets. The Genetic Algorithm adds to this hybridizing the property of total research like an initialization of the Fuzzy Neural Networks training algorithm which is based on an adapted version of the back propagation algorithm. The classes are represented as fuzzy sets with degrees of memberships. Rami J. and membership degrees and functions of fuzzy logic. (2005) present the pixel classification of biomedical image using neuro fuzzy approach. The idea is to combine three metaphors: Neural Networks.1 Related Works This section will present some approaches that have been developed and used in MRI images. The method showed high quality classification for images of simple components. Classified pixels are finally shown as three separate images each representing a set... The Neural Networks and Fuzzy Logic metaphors are coupled in one system called Fuzzy Neural Networks.Oweis et al.Benamrane et al. Fuzzy Logic and Genetic Algorithms in a hybrid system.

The perceptron is trained to produce a binary classification of each pixel as either a boundary or a non-boundary point. Subsequently. Chin-Ming Hong et al. Ian Middleton et al. (2003) reports on the combined use of a neural network(a multi layer perceptron.. (2006) propose a novel neuro fuzzy network which can efficiently reason fuzzy rules based on training data to solve the medical diagnosis problems. a novel cost function based on effectiveness is proposed for training that allows us to achieve dramatic improvements in segmentation performance. non-iterative selection of training examples. Ramiro Castellanos et al. Thereafter. (2000) have evaluated a segmentation technique for magnetic resonance (MR) images of the brain based on the adaptive fuzzy leader clustering (AFLC) algorithm. where the value of a vigilance parameter restricts the number of prototypes representing the feature vectors. MLP) and active contour model (‘snake’) to segment structures in magnetic resonance (MR) images. used to link the candidate boundary points into a continuous. as well as faster. Training sets are first developed using a lengthy iterative process.. Segmentation of MR images is formulated as an unsupervised vector quantization process. This study proposes a refined K-means clustering algorithm and a gradient-based learning rule to logically determine and adaptively tune the fuzzy . The inputs to the MLP are limited to normalized intensity values of the pixels from a (7 × 7) window scanned across the image.. The classifications produced using this cost function are sufficiently good since the binary image produced by the MLP can be post-processed using an active contour model to provide an accurate segmentation. The use of spatial coordinates as additional inputs to the MLP is then shown to provide an improvement in segmentation performance as quantified using the effectiveness measure (a weighted product of precision and recall). This approach performs vector quantization by updating the winning prototype of a competitive network through an unsupervised learning process. which are interpreted by the Fuzzy Neural Network of specification. the resulting binary (edge-point) image forms the external energy function for a snake. closed contour.9 Network detects the suspect regions.

The quantification is achieved by model-histogram fitting of probabilistic self-organizing mixtures and the segmentation by global consistency labeling through a probabilistic constraint relaxation network.. S. use of: iterative thresholding. The problem is formulated as distribution learning and relaxation labeling that may be particularly useful in quantifying and segmenting abnormal brain tissues where the distribution of each tissue type heavily overlaps. In the meanwhile. . one is the feature difference between neighboring pixels in the image. For all these groups many methods have been proposed in the literature but no one has been widely accepted in order to be used as a general method in clinics. (2003) show the neighborhood attraction based on the traditional fuzzy c-means (FCM) clustering algorithm to improve the segmentation performance. the other is the relative location of neighboring pixels. The new technique utilizes suitable statistical models for both the pixel and context images. Shen et al.. Yue Wang et al. In the category of threshold-based segmentation. (1997) present a probabilistic neural network based technique for unsupervised quantification and segmentation of the brain tissues from magnetic resonance image. The image techniques used for brain segmentation could be divided in the following groups: threshold-based segmentation. In this algorithm. Experimental results indicate that the proposed neuro fuzzy network with feature reduction can discover very simplified and easily interpretable fuzzy rules to support medical diagnosis. A histogram specification technique is applied on brain MR images before segmentation. statistical methods for brain segmentation and region growing methods. two factors of neighborhood attraction are considered. (1995) has surveyed and presented some segmentation methods that have been proposed in medical image community. this study also presents a feature reduction scheme based on the grey-relational analysis to simplify the fuzzy rules obtained from the employed neuro fuzzy network.10 membership functions for the employed neuro fuzzy network. Clarke et al.. Non-brain regions of all the MR images used in this paper have already been removed. The method can enhance the contrast between different brain tissues.

Since all these techniques have their advantages and drawbacks.11 histogram analysis and morphological operations are proposed.2 Artificial Neural Networks 2.5 X 10-9 g. There is something like 1010 to 1012 neurons in the human nervous system and each is capable of storing several bits of “information”. there are a lot of problems that researchers have to deal with in order to build an accurate segmentation system. the use of statistical methods in texture analysis has been proposed.1 Learning from Biology A small neuron is a small cell that receives electrochemical stimuli from multiple sources and responds by generating electrical impulses that are transmitted to other neurons or effectors cells. The segmentation problem widely encountered by researchers is the noise introduced with the acquisition of the image. so an average neuron weighs something less than 1. Statistical methods represent another important category in the segmentation process and most of the approaches proposed in this category use some statistical classifications combined with different image processing techniques in order to segment the MR images. 2. Some of these techniques are followed by some refinement of the segmented zones. the partial volume effect (when a pixel represents more than one kind of tissue type) and also some anatomical changes from one person to another.2. The blood vessels could also influence image that is taken and could introduce some noise.5kg. more research is demanded in this domain and maybe some combination of the already existing methods or some new image processing techniques will lead to better results. . The total weight of an average brain is 1. the overlapping intensities (different brain structures have different tissue characteristics which results in various signal intensities and these intensities could overlap). Also. So.

Neural activity is related to the creation of an internal electric potential called a membrane potential.1 below shows a simplified biological neuron. The activity of a neuron is measured by the firing frequency of the potential analogue spikes which it generates. a cell body. About 10% of neurons are input (afferent) and output (efferent).12 Neurons receive input from sensory or other types of cells and send outputs to other neurons or effectors organs such as muscles and glands. numerous dendrite links providing input connections from other neurons through synapses. the neuron “fires” by propagating a sequence of action potential spikes down the axon to either excite or inhibit other neurons. When the cumulative inputs raise the potential above a threshold value. They range from about 50 to a few hundred spikes per second. . Figure 2. in turn. they all share some common characteristics. and the time required for a stimulus to “traverse” a synapse is about 10ms during which the neuron cannot fire again. They are composed of nucleus. The remaining 90% are interconnected with other neurons which store information or perform various transformations on the signals being propagated through the networks. The connections are made through two general types of synapses. The pulses cause a chemical neurotransmitter substance to be released at the terminating synapses which. Neurons are complex cells that respond to electrochemical signals. can excite or inhibit other neurons. excitatory and inhibitory. The rate of pulse propagation ranges from about 5 to 125 ms-1. A single neuron may be connected to hundreds or even tens of thousands of other neurons. This potential may be increased or decreased by the input activity received from other cells through the synapses. Although many different types of neurons have been identified. and an axon trunk that carries an action potential output to other neurons through terminal links and synapses.

a typical ANN neuron or computing element is basically a comparator that produces an output when the cumulative effect of the input stimuli exceeds a threshold value. adaptive control. data compression. ANNs have been shown to be effective as computational processors for various tasks including pattern recognition in speech and visual image recognition. associative recall.13 Figure 2. multi sensor data fusion and noise filtering. Biological neurons or networks are able to process millions of input stimuli in milliseconds even though the processes are electrochemical in nature and propagate at relatively slow milliseconds rates. combinational problem solving. classification. 2.2 Generic Model of Neural Network Although ANN architectures differ in several characteristic ways.1: Biology Neuron Artificial Neural Networks (ANNs) are imitation of simplified models of the central nervous system. They are networks of highly interconnected neural computing elements that have the ability to respond to input stimuli and to learn to adapt to the environment.2. a sort of filter which is part of the linkage connecting the input to the . modeling and forecasting. Each input link i has an associated external input signal or stimulus xi and a corresponding weight wi.

1).14 neuron. Figure 2. The outputs from the ANN can also be real-valued or binary or bipolar. but are usually assumed to be real (positive for excitatory and negative for inhibitory links). Normally. act to either increase (excitatory) or decrease (inhibitory) the input signals to the neuron. The xi values can be real (+ or -). The weights can also be binary or real-valued.3 Bias and Threshold A bias acts exactly like a weight and it is considered as a connection and its activation function is always 1.2. binary (0.2: Generic model of Artificial Neural Network The weights. A bias is often used to increase . It is adapted similar to the way a weight is adapted according to the learning rule of the neural network. where net is the cumulative input stimuli to the neuron and f is typically a nonlinear function of net. or bipolar (-1. net = x1 w1 + x 2 w2 + x3 w3 = ∑ xi wi i =1 3 (2.2. 1). The neuron behaves as an activation or mapping function f(. net is often taken as the weighted sum of the inputs and f is typically a monotonic non-decreasing function of net and this can be summarized in the equation below.1) 2.) producing an output y = f(net). which model the synaptic neural connections in biological nets. This is shown in Figure 2.

mostly chosen from [0. Some of the units are termed input units: these receive signals not from other units. Units that do not transmit signals to other units are termed output units. such as to improve convergence. 2.1} or {-1. Multilayered models generally use a sigmoid activation function.4 Activation Functions In most neural network models. and therefore have continuous activations. some models of neural network do not use any bias signals.3 below.1}. In thermodynamic models a linear threshold function is normally used.2.5 Network A neural network is formed when we place units at the vertices of the directed graph. It is quite similar to a bias but not adaptable. but instead they take their signals from the outside environment. .2. This is illustrated in the Figure 2. A threshold or normally symbolized as θ. is a value that is used to make some form of decisions in an artificial neural network such that the neural network will fire or unfire. 1].15 the signal levels in neural network. with the arcs of the digraph representing the flows of signals between units. The network is said to be a feed-forward network when the units can be labeled with integers in such a way that there is a connection from the computation unit labeled i to the computation unit labeled j. 1] or [-1. However. The choice of the activation function strongly depends on the neural network model. 2. then i < j and it also can be in multilayer networks. and the activation values are chosen from {0. one of the above mentioned activation functions is used.

propagated through the network. This total error is calculated anew after each epoch. until it has reached its inoperative mode.16 Figure 2. The error value (e) is determined and added to the total error E. As a stop criterion for the learning process. then the weights are changed in a way that a reduction of the absolute value of error.6 Learning Phase Basically. A pair of patterns of the learning problem is chosen and the input i presented to the neural network. iii. A supervised learning algorithm can be divided into five steps: i. The input is then. If e ≠ 0. there are two ways of learning in artificial neural network which are supervised and unsupervised learning. The overall concept of learning is to change the weights in the neuron in such a way that the error value is minimized for each pair of patterns of the fixed learning problem. . The learning process stops when the total error E is close enough to 0. generally a total error E is used that is usually chosen to be the sum of the single error values. iv. The output determined by the network is compared to the target pattern. ii.3: Feedforward Network 2. An epoch is a complete run through the learning problem such that each pair of input / output patterns is processed once by the network using the learning algorithm. or when the network is obviously not able to solve the learning problem.2.

If the current epoch is not yet over. this learning procedure cannot always guarantee convergence. This means that the minimized error is not reached and the learning process has failed. E is checked and when the stop condition reached. In unsupervised learning algorithm. This theory is a result of the insufficiency of Boolean Algebra to many problems of the real world. because it is equivalent to a local heuristic search procedure. On the other hand. unsupervised learning algorithm works on a free learning problem which only contains input patterns. the units compete with each other and the principle of winner-takes-all applied. There are however. total error. it will repeat the general steps above until it converges or reaches the stop condition. 2. The most common supervised learning algorithms involve approximation of a gradient descent method and try to reduce the total error E to zero.17 v. the weights would have been adapted in such a manner such that similar patterns are clustered into a particular node. The algorithm stops when it reaches a local minimum.3 Fuzzy Logic Systems The theory of Fuzzy Logic was first raised by the mathematician Lotfi A. It is trained without teaching signals or targets and it is only supplied with examples of the input patterns that it will solve eventually. then the procedure continues with step (i) and the value of E is kept. As most of the information in the real world is . then. However. When an epoch is over. some disadvantages of this technique as the local minimum is not the global minimum. This type of learning usually has an auxiliary cost function which needs to be minimized and the weights are modified where a cost function is minimized. Else. Zadeh in 1965. the learning process has ended successfully. for this kind of learning procedure the network is also supposed to map similar input patterns to output that are similar to each other. However. At the end of the learning phase.

With any given system. confused or vague. The heart of a fuzzy system is a knowledge base consisting of the so called fuzzy IF-THEN rules.3. What makes fuzzy nice is the "naturalness" of its approach and not its far-reaching complexity. which are available in the Fuzzy Logic Toolbox. • Fuzzy logic is flexible. the word Fuzzy is defined as blurred.1 Advantages of Fuzzy Logic Below are some reasons why we need to use fuzzy logic rather than linear system: • Fuzzy logic is conceptually easy to understand. • Fuzzy logic can be built on top of the experience of experts. most things are imprecise even on careful inspection. it's easy to massage it or layer more functionality on top of it without starting again from scratch. but more than that. A fuzzy system can be created to match any set of input-output data. indistinct. • Fuzzy logic can model nonlinear functions of arbitrary complexity.18 imprecise. imprecisely defined. . Fuzzy systems are knowledge based or rule based systems. This process is made particularly easy by adaptive techniques like Adaptive Neuro-Fuzzy Inference Systems (ANFIS). Everything is imprecise if you look closely enough. According to the Oxford English Dictionary. 2. A fuzzy IF-THEN rule is an IF-THEN statement in which some words are characterized by continuous membership functions. • Fuzzy logic is tolerant of imprecise data. and one of humans' greatest abilities is to effectively process imprecise and "'fuzzy" information. The mathematical concepts behind fuzzy reasoning are very simple. Fuzzy reasoning builds this understanding into the process rather than tacking it onto the end.

Natural language.3. The basis for fuzzy logic is the basis for human communication. We are generally unaware of this because ordinary language is. • Fuzzy logic is based on natural language. When should you not use fuzzy logic? The safest statement is the first one made in this introduction: fuzzy logic is a convenient way to map an input space to an output space. fuzzy logic is easy to use. 2. that which is used by ordinary people on a daily basis. impenetrable models. However. 1997]. use it. If a simpler solution already exists. • Fuzzy logic can be blended with conventional control techniques. If it's not convenient. This observation underpins many of the other statements about fuzzy logic. The last statement is perhaps the most important one and deserves more discussion. do a fine job without using fuzzy logic.2 Disadvantages of Fuzzy Logic Fuzzy logic is not a cure-all. . which take training data and generate opaque. has been shaped by thousands of years of human history to be convenient and efficient. Fuzzy systems don't necessarily replace conventional control methods. fuzzy logic lets you rely on the experience of people who already understand your system.use common sense when it is implemented and the right decision will probably been made. something we use every day. if time is taken to become familiar with fuzzy logic. Since fuzzy logic is built atop the structures of qualitative description used in everyday language. Many controllers. a very powerful tool can be proved for dealing quickly and efficiently with imprecision and nonlinearity. In many cases fuzzy systems augment them and simplify their implementation. Sentences written in ordinary language represent a triumph of efficient communication. try something else.19 In direct contrast to neural networks. [Jang and Sun. Fuzzy logic is the modification of common sense . for example. of course.

. into real numbers in [0. Such sets are also called crisp sets. small and others. This function is a generalization of that in equation 2. A : X → [0..4 below.. so two values of truth. each membership function maps elements of a given universal base set X which is itself a crisp set. Therefore... The notation for the membership function.3 and called a membership function... such as big. µA(x) of a fuzzy set A is as equation 2.2) If these elements ai (i = 1.4) Each fuzzy set is completely and uniquely defined by one particular membership function. a n } (2. The membership of a fuzzy set is described by this membership function µA(x) of A which associates to each element xo = X a grade of membership µA(xo)..n) of A are together a subset of the universal base set X. In contrast to classical set theory a membership function µA(x) of a fuzzy set can have in the normalized closed interval [0 1]. a 2 .1]. ..3 Fuzzy Sets In the classical set theory a set can be represented by enumerating all its elements using A = {a1 .4 shows the differences in the crisp and fuzzy sets. a3 .1] (2.. Figure 2. the set A can be represented for all elements x∈ X by its characteristic function µ A (x) =  1 0 if x ∈ A otherwise (2..3.20 2... Consequently symbols of membership functions are also used as labels of the associated fuzzy sets. Non-crisp sets are called fuzzy sets.3) In classical set theory µA(x) has only the values 0 (``false'') and 1 (``true''). for which also a characteristic function can be defined.

3.3.4: Membership functions of a crisp set and a fuzzy set 2.5: Membership grades of a fuzzy set 2. whereby for µA(x) a large class of functions can be taken. This is shown in Figure 2. .21 x Figure 2.1]. In some cases.5. The grade of membership µA(x) of a membership function describes which grade it belongs to in the fuzzy set A. rule base and defuzzification. This value is in the unit interval [0. such as triangular or trapezoidal functions. x Figure 2. inference engine.4 Membership functions The membership function µA(x) describes the membership of the elements x of the base set X in the fuzzy set A. Reasonable functions often used are linear functions. a fuzzy system is drawn as a black box with some inputs and an output.6 shows the components of a fuzzy system. Figure 2.5 Components of a fuzzy system There are basically four components in the fuzzy logic system which is fuzzification.

6: Components of a Fuzzy Logic System The first block inside the controller is fuzzification.22 Figure 2. The fuzzification block thus matches the input data with the conditions of the rules to determine how well the condition of each rule matches that particular input instance. µ B ( x)}. It is the heart of the fuzzy system in the sense that all other components are used to implement these rules in a reasonable and efficient manner. all rules in the fuzzy rule base is combined into a single fuzzy relation which is then viewed as a single fuzzy IF-THEN rule. Inference engine simply implies the combination of certain rules into a mapping from a fuzzy rule base to get some output corresponding to the inferred rules. A fuzzy rule base consists of a set of fuzzy IF-THEN rules.5) . Fuzzification is used to convert each piece of input data to degrees of membership by a lookup in one or several membership functions. Most common type of inference used is shown below:- • The fuzzy intersection operator ∩ (fuzzy AND connective) applied to two fuzzy sets A and B with the membership functions µA(x) and µB(x) is µ A∩ B ( x) = min{µ A ( x). In composition based inference. There is a degree of membership for each linguistic term that applies to that input variable. There are two ways to infer with a set of rules which are composition based inference and individual rule based inference. x∈ X (2.

It is not a unique operation as different methods are possible. Defuzzification is used to transform fuzzy information into a single value. The most common are centre of gravity (COG). Simple computation of fuzzy inference system is also one of the issues that need to be considered besides the intuitive issue and some inference need special properties. weight average and mean-max membership method. there are three criteria of choosing the inference methods which are intuitive appeal. the choice should make sense from an intuitive point of view. The value inferred by the fuzzy rule base is in the fuzzy value or in other words. In general. For intuitive appeal. x∈ X (2.7) In individual rule base inference. . for every possible value µ. maximum membership. centre of singleton (COS). one gets a grade of membership that describes to what extent this value µ is reasonable to use. computational efficiency and special properties. x∈ X (2. each rule in the fuzzy rule base determines an output fuzzy set and the output of the whole fuzzy inference engine is the combination of many individual fuzzy sets.6) • The fuzzy complement operator (fuzzy NOT operation) applied to the fuzzy sets A with the membership functions µA(x) is µ A ( x) = 1 − µ A ( x). µ B ( x)}. The combination can be taken either by union or by intersection.23 • The fuzzy union operator ∪ (fuzzy OR connective) applied to two fuzzy sets A and B with the membership functions µA(x) and µB(x) is µ A∪ B ( x) = max{µ A ( x).

6 Fuzzy Logic Graphical User Interface (GUI) There are five primary GUI tools for building.24 2. editing. and observing fuzzy inference systems in the Fuzzy Logic package: 1.3.7: Fuzzy Logic Graphical User Interface (GUI) . Fuzzy Inference System (FIS) Editor Membership Function Editor Rule Editor Rule Viewer Surface Viewer Figure 2. 5. 2. 4. 3.

or the number of membership functions is too big.25 The FIS Editor handles the high-level issues for the system. Fuzzy Logic Toolbox software does not limit the number of inputs. They are strictly read-only tools. The Membership Function Editor is used to define the shapes of all the membership functions associated with each variable. as opposed to editing. However. It computes a compact and optimal description of the data set. The first principal component is the combination of variables that explains the greatest amount of variation. and the last of these variables can be removed with minimum loss of real data. Essentially. The uncorrelated variables are linear combinations of the original variables. the FIS. a set of correlated variables are transformed into a set of uncorrelated variables which are ordered by reducing variability. If the number of inputs is too large. it can show (for example) which rules are active. The Rule Editor is for editing the list of rules that defines the behavior of the system. The Rule Viewer and the Surface Viewer are used for looking at. The Surface Viewer is used to display the dependency of one of the outputs on any one or two of the inputs-that is. The main use of PCA is to reduce the dimensionality of a data set while retaining as much information as is possible. the number of inputs may be limited by the available memory of the machine.4 Principal Component Analysis (PCA) Principal Components Analysis (PCA) is a multivariate procedure which rotates the data such that maximum variability is projected onto the axes. The second principal component defines the next . The Rule Viewer is a MATLAB technical computing environment based display of the fuzzy inference diagram shown at the end of the last section. Used as a diagnostic. then it may also be difficult to analyze the FIS using the other GUI tools. it generates and plots an output surface map for the system. or how individual membership function shapes are influencing the results. 2.

The second axis contains the maximum amount of variation orthogonal to the first. the second new variable contains the maximum amount of variation unexplained by the first and orthogonal to the first. It can be viewed as finding a projection of the observations onto orthogonal axes contained in the space defined by the original variables. It can be viewed as a rotation of the existing axes to new positions in the space defined by the original variables. The first new variable contains the maximum amount of variation. there will be no correlation between the new variables defined by the rotation. The criteria being that the first axis "contains" the maximum amount of variation. etc. The third axis contains the maximum amount of variation orthogonal to the first and second axis and so on until one has the last new axis which is the last amount of variation left. There can be as many possible principal components as there are variables. . In this new rotation. or "accounts" for the maximum amount of variation.26 largest amount of variation and is independent to the first principal component.





Basically, two main phases are employed in this project which is theoretical phase and practical phase. Theoretical phase involves the process of reading and understanding, reviewing of theories, studying and surveying the ANFIS model, classification and detection techniques on medical image fields.

On the other hand, the practical phase involves medical image processing, neuro fuzzy model coding, simulation, assumptions, adaptation and training of the neuron and etc by using MATLAB.


Proposed Method

There are 7 stages involved in the proposed model which starts from the data input to output. The first stage is the image processing system. Basically in image processing system, image acquisition and enhancement are steps that need to be done. In this project, these two steps are skipped and all the images are collected from available resources. The proposed model requires converting the image into a format capable of being manipulated by the computer. The MR images are converted into matrices form by using MATLAB. Then, the neuro fuzzy model is developed using MATLAB programming. After the neuro fuzzy model is successfully developed, the classification of the MR images starts.

28 Symptoms detection phase will follow once the output from the classification technique is done and lastly, performance based on the result will be analyzed at the end of the development phase. The proposed brain MR images classification and detection method is shown in Figure 3.1.

MR Image Input


Data Form

Neuro Fuzzy Model

Classification & Detection


Performance Analysis Figure 3.1: The proposed system


Transformation of Image to Data Form

The MR image will obtain and convert it to data form (encoded information that can be stored, manipulated and transmitted by digital devices) in MATLAB environment such as basic arithmetic operations (addition, subtraction, and multiplication) and indexing, including logical indexing reshaping, reordering, and concatenating. The basic data structure in MATLAB is the array, an ordered set of real or complex elements. This object is naturally suited to the representation of images, real-valued ordered sets of color or intensity data. MATLAB stores most images as two-dimensional arrays (i.e., matrices), in which each element of the

29 matrix corresponds to a single pixel in the displayed image. (Pixel is derived from picture element and usually denotes a single dot on a computer display.) For example, an image composed of 200 rows and 300 columns of different colored dots would be stored in MATLAB as a 200-by-300 matrix. This convention makes working with images in MATLAB similar to working with any other type of matrix data, and makes the full power of MATLAB available for image processing applications. An intensity image is a data matrix, whose values represent intensities within some range. MATLAB stores an intensity image as a single matrix, with each element of the matrix corresponding to one image pixel. The matrix can be of class double, uint8, or uintl6.

A simple program (shown in Appendix) is created to transform images. dot of pixel labelling and also region of interest extraction. Initially, this program asked the user to insert normal brain image as the prior image and then insert an abnormal brain image as a diagnostic image. Both images are converted to matrix format automatically while the image being read in the MATLAB environment. Both images are obtained in the RGB format. An RGB image is stored in MATLAB as an m-by-n-by-3 data array that defines red, green, and blue color components for each individual pixel. If it is difficult to manipulate, it must be converted to intensity format. An intensity image is a data matrix, whose values represent intensities within some range. MATLAB stores an intensity image as a single matrix, with each element of the matrix corresponding to one image pixel. The elements in the intensity matrix represent various intensities, or gray levels, where the intensity 0 usually represents black and the intensity 1, 255 or 65535 usually represent full intensity, or white. Both intensity of normal brain and abnormal brain images are converted to binary images as shown in Figure 3.2.

The 'bwarea' function is used to convert the area to binary image. The area is a measure of the size of the foreground of the image. In other words, the area is the number of on pixels in the image. Comparison between the areas of the normal and the abnormal brain in a binary image is a technique used to make sure the size of both brain images are the same.

Figure 3.30 Figure 3. Under this coordinate system.3: Pixel coordinate system For pixel coordinates. ordered from top to bottom and left to right.2: Convert brain image to binary image 3. as illustrated in Figure 3. the image is treated as a grid of discrete elements. every dot of pixel will be labeled in different coordinates. Generally in the Matlab environment. There is a one-to-one correspondence between pixel coordinates and the coordinates MATLAB uses for matrix subscripting. while the second component c (the column) increases to the right. Pixel coordinates are integer values and range between 1 and the length of the row or column. the first component r (the row) increases downwards. This correspondence makes the relationship .3.4 Pixels Labeling In succession. the most convenient method for expressing locations in an image is to use pixel coordinates.

For instance. advanced coordinate system will be created. By using the existing pixel coordinate system. however pixel in the first row.5 Region of Interest Extraction The region of interest is determined by subtracting the normal brain image with abnormal image brain image as shown in Figure 3.5. every dot of pixel coordinate in a single number. second column is stored in the single number 2 and so on. second column is stored in the matrix element (5. It means that. For example. the data for the pixel in the fifth row. Figure 3.4: Advanced coordinate system for matrix 16 by 16 3.31 between an image's data matrix and the way the image is displayed easy to understand. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 Figure 3. first column is stored in the single number 1.4 shows the advanced coordinate system for matrix 16 by 16. . pixel in the first row. 2).

where the intensity 0 usually represents black and the intensity 1. or 65535 usually represents full intensity. the principal component analysis (PCA) is used as a feature extraction algorithm. The principal component analysis (PCA) is one of the most successful techniques that have been used in image recognition and compression. or grey levels. or white.6 Feature Extraction using Principal Component Analysis (PCA) In this project.32 Normal Brain Abnormal Brain Region of Interest Figure 3. 255. . The purpose of PCA is to reduce the large dimensionality of the data.5: Location of tumor The elements in the intensity matrix represent various intensities. 3.

In order to extract PCA features of first convert the image into a pixel vector Φ1 by concatenating each of the M rows into a single vector. N columns). In the testing phase. In this project. these feature vectors ωi are calculated and stored. j is computed the similarities between ωj and all of the feature vectors ωi’s in the training set are computed. . The similarity between feature vectors is computed using Euclidean distance. the feature vector ωj of the test image using PCA.6. feature vectors are extracted for each image in the training set. the PCA algorithm is used as a dimensionality reduction technique which transforms the vector Φ1 to a vector ω1 which has a dimensionality d where d << M x N. If i = j. In order to identify the test image j. it means that the MR image j has correctly identified. The memory of an MR image recognizer is generally simulated by a training set. In the training phase. the training database consists of a set of MR images. For each training image i. dimensionality) of the vector Φ1 will be M x N. x N (M rows. The identity of the most similar ωi is the output of the image recognizer. The length (or. Schematic diagram of the MR image recognition system that is implemented is shown in Figure 3. otherwise if i ≠ j. In this project. it means that the MR image j has misclassified. Thus. Let 1 be a training image of image 1 which has a pixel resolution of M 1. the task of the MR image recognizer is to find the most similar feature vector among the training set to the feature vector of a given test image.33 MR image recognition systems find the identity of a given test image according to their memory.

6: Schematic diagram of a MR image recognizer. This project uses the ANFIS Editor GUI menu bar to load a FIS training initialization.34 Figure 3. . and then save the trained FIS. The loaded data is plotted on the plot region. 3. a training data set that contains the desired input/output data pairs of the targeted system is modeled. Load data (training) by selecting the appropriate radio buttons in the Load data portion of the GUI and then clicking Load Data.7 Data Modeling Eventually. The coordinated abnormal tissues which are in the matrix form are presented to ANFIS for training (estimating) membership function parameters. They fully represent the features of the data that the trained FIS intends to model. From this GUI: l.

. This research uses 0 as training epochs and 40 as training error tolerance. and. Therefore. The training data is a required argument to the ANFIS Editor GUI. the number of rows of the training data is equal to the number of training data pairs. Each row starts with an input vector and is followed by an output value. the number of columns training data is equal to the number of inputs plus one. Choose the FIS model parameter optimization method backpropagation or a mixture of backpropagation and least squares (hybrid method). Train the FIS model by clicking the Train Now button. Each row of the training data is a desired input/output pair of the target system to be modeled. View the FIS model output versus the training. 6. Figure 3.7 shows the training data set for this research. and testing data output by clicking the Test Now button. This research uses the default hybrid method as the FIS model parameter optimization method. 5. This training adjusts the membership function parameters and plots the training error plot(s) in the plot region. since there is only one output. 7. Choose the number of training epochs and the training error tolerance. checking. 3. 4. View the FIS model structure once an initial FIS has been generated or loaded by clicking the Structure button. This function plots the rest data against the FIS output in the plot region. Generate an initial FIS model or load an initial FIS model using the options in the Generate FIS portion of the GUI.35 2.

To evaluate the output of a fuzzy system for a given input. If input is an M-by-N matrix. where M represents the number of input values specified above.7: Training Data Set After data modeling by ANFIS. where N is a number of input variables.36 Input vector Output Figure 3. • Input: a number or a matrix specifying input values. Figure 3. read this fuzzy inference system from an anfis. use the function evalfis. • • Fismat: a FIS structure to be evaluated. output.fis file on the disk and bring the resulting file into workspace. and L is the number of output variables for the FIS. then evalfis takes each row of input as an input vector and returns the M-by-L matrix to the variable. The ‘evalfis’ has the following arguments. where each row is an output vector and L is the number of output variables. Output: the output matrix of size M-by-L.8 shows the Basic flow diagram of computations or algorithm in ANFIS: .

it is advisable to know its data management system.8.1 Data Management and Preparation As this project fully use the MATLAB software. usually the . all files are placed in a folder included in the MATLAB working path. To work with m-files and data. or typing ‘path’ in the prompt. The working path is accessible by clicking the ‘Path Browser’ button in the MATLAB command window. The most practical solution for beginners is to place files in the folder ‘…/MATLAB/Work’. Any file located outside this path will be invisible to MATLAB.8: Basic flow diagram of computations in ANFIS 3.37 Initialize the fuzzy system Use genfis1 command Give other parameters for learning Important are: Number of iterations (epochs) Tolerance (error) Start learning process Use anfis command Stop when tolerance is achieved Validation data Use evalfis command Figure 3. The working path is a collection of folders where MATLAB searches files when these are called.8 System developing 3.

The main program is developed in 4 phases which are input phase. 4 main input files are needed. the folder where MATLAB preferentially loads or saves data). is used to check the model created. checking and application. Training data means that the data is used for ANFIS training and checking on the other hand. Application data is a collection of untrained data needed in testing the accuracy of the model. checking data . In this package development. It has a total of 50 data with 3 inputs and 1 output.e. classification and FIS phase.8. An error will occur if m-files files are not in its directory. The data is divided into 3 partition which are training. ANFIS phase. Training data has 20 patterns.8. and only edit the MATLAB working path if either this folder is absent or it is not the current directory. It means that the current directory is not in the folder. MATLAB is case sensitive. output phase (application and detection) and is explained in details below. any data file or m-file should be named properly. checking takes around 5 patterns and the rest which is 25 patterns are used for application purposes.3 Input Phase In this phase. the user is required to enter the files that have been partitioned accordingly by keying the file name in the space provided. Thus. The MATLAB working path is easily edited with the ‘Path Browser’ tool. But we recommend beginners to work with the ‘…/MATLAB/Work’ folder as the working directory. 3. They are training data file.38 default current directory (i.2 Data Mining The data are provided from the transformation MR images into data form as explained before. 3.

4 FIS Phase After the input phase.39 file. User also is required to enter the total number of cluster and epochs required for training purposes.8. On the other hand. It is always an easy way to paste data from a spreadsheet in the Windows Notepad and saving it under extension ‘dat’. relevant data files will be loaded for Fuzzy Inference System (FIS) building. target data file and application data file including expected result data file. Figure 3. arranged in rows and columns. the checking file that is used for model checking and validation should contain data and training target together.9: Partial of input phase 3. Genfis1 function . Figure 3. A convenient format to work with MATLAB is ASCII. For the training data file. All data and m-files should be placed in a folder included in the MATLAB working path. it is advisable to contain the data only and not combined together with training target. Genfis1 basically generates initial fuzzy inference system for ANFIS training by first applying classification technique on the data.9 shows a partial of the input phase and full coding can be accessed in Appendix. The fuzzy inference system is summarized in a function called genfis1. Application data purely contain the untrained data and is subjected to test the accuracy of the model.

5 ANFIS modeling phase This phase is fully the training of ANFIS model. target data and number of membership function required. The second column shows the training error and third column indicates the checking error.10 shows a sample of generated fuzzy inference. Figure 3. Result from training error and checking error will be calculated depending on the step size and is in matrix form. Output from genfis1 is considered as initial information for building the fuzzy inference system which is useful for ANFIS training.40 requires 3 main inputs which are training data without target. Figure 3. Figure 3.8.10: FIS generated thru genfis1 3.11.12 shows the errors regarding to the number of epoch and the total time required for ANFIS training. The data such as number epoch and fuzzy inference system (FIS) is used to train and validate the ANFIS model. This is accomplishes by extracting a set of rules that models the data behavior. Information regarding the ANFIS model will prompt at the early of the process as shown in Figure 3. .

41 Figure 3.11: Information of the ANFIS model Figure 3.12: ANFIS training errors and total time training .

User is required to enter the application file name and also the expected result. the class of the abnormal tissue can be determined.13 shows an example of the result.13: Sample of classification results . Figure 3. Figure 3.8.6 Output phase The output phase involves the application of the model to observe how well the model reacts to the untrained data and also the detection of abnormal tissue based on the data itself. By making comparison between the expected result and output from the ANFIS model.42 3.

we need to replace the functions used in the fuzzy system (like min and max) by differentiable functions or do not use a gradient-based neural learning algorithm but a better-suited procedure. This cannot be applied directly to a fuzzy system. because it is based on a fuzzy system reflecting vague knowledge.CHAPTER 4 NEURO FUZZY MODELING AND SOFTWARE IMPLEMENTATION 4. . An even more important aspect is that the system should always be interpretable in terms of fuzzy if-then rules. because the functions used to realize the inference process are usually not differentiable. neural network learning algorithms are usually gradient descent methods.1 Neuro Fuzzy Model A neuro fuzzy system is a combination of neural network and fuzzy systems in such a way that neural network or neural network algorithms are used to determine the parameters of the fuzzy system. A common way to apply a learning algorithm to a fuzzy system is to represent it in a special neuralnetwork-like architecture. Then a learning algorithm such as back propagation is used to train the system. In order to realize the system. The idea of a neuro fuzzy system is to find the parameters of a fuzzy system by the means of learning methods obtained from neural network. However. This means that the main intention of neuro fuzzy approach is to create or improve a fuzzy system automatically by means of neural network methods.

for example the membership functions bounds. . It also approximates an ndimensional (unknown) function that is partially given by the training data. A neural network is used to automatically tune the system parameters. The fuzzy rules encoded within the system represent vague samples. a neuro fuzzy system should not be seen as a kind of (fuzzy) expert system. It is possible both to create the system out of training data from scratch. It can be viewed as a special kind of feed forward neural network. Fuzzy sets are encoded as (fuzzy) connection weights. and it has nothing to do with fuzzy logic in the narrow sense. and can be viewed as vague prototypes of the training data. and to initialize it from prior knowledge in the form of fuzzy-rules. Besides. and causes only local modifications in the underlying fuzzy system. This results in constraints on the possible modification of the system’s parameters. leading to improved performance without operator invention. The learning process is not knowledge-based.2 Adaptive neuro-fuzzy inference system (ANFIS) A neuro-fuzzy (ANFIS) system is a combination of neural network and fuzzy systems in such a way that neural network is used to determine the parameters of fuzzy system. ANFIS largely removes the requirement for manual optimization of the fuzzy system parameters. but datadriven. Generally. 4. The units in this network use t-norms or t-conorms instead of the activation functions normally used in neural networks. The (heuristic) learning procedure operates on local information. The learning procedure of a neural-fuzzy system takes the semantical properties of the underlying fuzzy system into account. A neuro fuzzy system is a fuzzy system that is trained by a learning algorithm (usually) derived from neural network theory.44 Modern neuro fuzzy systems are often represented as multilayer feed forward neural network. a neural-fuzzy system can always be interpreted as a system of fuzzy rules.

1 illustrates the reasoning mechanism for this Sugeno model where it is the basis of the ANFIS model. Figure 4. Does not require prior human expertise o uses membership functions plus o desired dataset to approximate Greater choice of membership functions to use. . 1993). However. qi and ri are the design parameters that are determined during the training process. the system is built by fuzzy logic definitions and is then refined using neural network training algorithms. 4. two fuzzy if-then rules based on a first order Sugeno model are considered: Rule 1: If (x is A1) and (y is B1) then (f1 = p1x + q1y + r1) Rule 2: If (x is A2) and (y is B2) then (f2 = p2x + q2y + r2) where x and y are the inputs. fi are the outputs within the fuzzy region specified by the fuzzy rule. 1992. To present the ANFIS architecture.45 The neuro fuzzy system with the learning capability of neural network and with the advantages of the rule-base fuzzy system can improve the performance significantly and can provide a mechanism to incorporate past observations into the classification process. using a neuro fuzzy scheme. Very fast convergence time. pi. In neural network the training essentially builds the system. Some advantages of ANFIS are: • • Refines fuzzy if-then rules to describe the behaviour of a complex system.2. Ai and Bi are the fuzzy sets. Such framework makes the ANFIS modeling more systematic and less reliant on expert knowledge.1 ANFIS Architecture The ANFIS is a fuzzy Sugeno model put in the framework of adaptive systems to facilitate learning and adaptation (Jang.

Figure 4.2) . (4.46 Figure 4. which are given by: O1.i = µ Ai (x) O1. all the nodes are adaptive nodes. 4.i = µ Bi−2 (y) i = 1.1) (4. in which a circle indicates a fixed node.2: ANFIS architecture. whereas a square indicates an adaptive node. In the first layer. Adaptive neuro fuzzy inference system basically has a 5 layer architectures and each of the function is explained in detail later.1: A two-input first-order Sugeno fuzzy model with two rules The ANFIS architecture to implement these two rules is shown in Figure 4. The outputs of layer 1 are the fuzzy membership grade of the inputs.2. 2 i = 3.

the nodes are also fixed nodes labeled by N.47 where µ Ai(x). bi and ci are the parameters of the membership function. µ Ai (x) is given by: µ A ( x) = 1 x − ci 1+ ai 2b (4. The output of each node in this layer is simply the product of the normalized firing strength and a first order polynomial (for a first order Sugeno model). For example. wi + w2 i = 1. Thus. In the fourth layer. 2 (4.4) which are the so-called firing strengths of the rules. the output of this layer is given by: O4.i = wi f i = wi ( pi x + q i y + ri ) i = 1. 2. The outputs of this layer can be represented as: O2. µ Bi−2(y) can adopt any fuzzy membership function.6) . They are labeled with π.5) which are the so-called normalized firing strengths. The output of this layer can be represented as: O3. the nodes are fixed nodes.2. governing the bell shaped functions accordingly. (4. to indicate that they play a normalization role to the firing strengths from the previous layer. In the second layer.i = wi = wi . indicating that they perform as a simple multiplier. (4. if the bell shaped membership function is employed.i = wi = µ Ai (x)µ Bi (y) i = 1.3) where ai. In the third layer. the nodes are adaptive.

This node performs the summation of all incoming signals. These parameters are the so-called premise parameters. there are also three modifiable parameters {pi. each of which is governed by a fuzzy if-then rule. In the fourth layer. 1992. bi. ri}.48 In the fifth layer. pertaining to the first order polynomial. . which are related to the input membership functions. the premise part of a rule defines a fuzzy region. Hence. ci}. In the first layer. Figure 4. while the consequent part specifies the output within the region.3 shows the variation in the Sugeno model that is equivalent to a two-input first-order Sugeno fuzzy model with nine rules. In other words. qi.4 illustrates how the two dimensional input space is partitioned into nine overlapping fuzzy regions.7) It can be observed that there are two adaptive layers in this ANFIS architecture. Figure 4. namely the first and the fourth layers. where each input is assumed to have three associated MFs. there is only one single fixed node labeled with ∑. there are three modifiable parameters {ai. These parameters are the so-called consequent parameters (Jang. the overall output of the model is given by: O 5.i = ∑ wi f i = i ∑wf ∑w i i i i i (4. 1993).

bi and ci of the membership function are fixed.2 Learning algorithm of ANFIS The task of the learning algorithm for this architecture is to tune all the modifiable parameters. ci} and {pi.2.4: The input space that are partitioned into nine fuzzy regions 4. qi. to make the ANFIS output match the training data. When the premise parameters ai.3: Two-input first-order Sugeno fuzzy model with nine rules Figure 4. ri}.49 Figure 4. the output of the ANFIS model can be written as: . namely {ai. bi.

q2 and r2. the search space becomes larger and the convergence of the training becomes slower. 1992. The gradient descent method (backward pass) is used to adjust optimally the premise parameters corresponding to the fuzzy sets in the input domain. the backward pass starts immediately.9). The least squares method (forward pass) is used to optimize the consequent parameters with the premise parameters fixed.10) After rearrangement. q1.1 summarizes the activities in each pass. The output error is used to adapt the premise parameters by means of a standard backpropagation algorithm. It has been proven that this hybrid algorithm is highly efficient in training the ANFIS (Jang. The least squares method can be used to identify the optimal values of these parameters easily.11) which is a linear combination of the modifiable consequent parameters p1. (4.8) yields: f = w1 f 1 + w2 f 2 (4. (4. (5) into Eq. A hybrid algorithm combining the least squares method and the gradient descent method is adopted to solve this problem. When the premise parameters are not fixed. Table 4.50 f = w1 w2 f1 + f2 w1 + w2 w1 + w2 (4. the output can be expressed as: f = ( w1 x) p1 + ( w1 y )q1 + ( w1 )r1 + ( w2 x) p 2 + ( w2 y )q 2 + ( w2 )r2 (4.8) Substituting Eq. it becomes: f = w1 ( p1 x + q1 y + r1 ) + w2 ( p 2 x + q 2 y + r2 ) (4. p2. r1. The output of the ANFIS is calculated by employing the consequent parameters found in the forward pass. . The hybrid algorithm is composed of a forward and a backward pass. 1993).9) Substituting the fuzzy if-then rules into Eq. Once the optimal consequent parameters are found.

To achieve good generalization of unseen data. the size of the training data set should be at least as big as the number of modifiable parameters in ANFIS. Gradient descent and Backpropagation algorithms are used to adjust the parameters of membership functions (fuzzy sets) and the weights of defuzzification (neural networks) for fuzzy neural networks.3 ANFIS Classifier Both neural network and fuzzy logic are universal estimators. For premise parameters that define membership functions. The adaptation of back propagation to structured network so as to automate fuzzy control parametric tuning is utilized from NN. The ANFIS is a FIS implemented in the framework of an adaptive fuzzy neural network. The advantage of fuzzy is that prior knowledge is represented into a set of constraints to reduce the optimization research space. ANFIS applies two techniques in updating parameters. It combines the explicit knowledge representation of a FIS with the learning power of ANNs. They can approximate any function to any prescribed accuracy.2.51 Forward pass Premise parameters Consequent parameters Signals Fixed Least-squares estimator Node outputs Backward pass Gradient descent Fixed Error signals Table 4. The objective of ANFIS is to integrate the best features of fuzzy systems and neural network. For consequent parameters that define the coefficients of each equation. Functionally there are almost no constrains on the node functions of an adaptive network except for the requirement of piecewise differentiability. ANFIS employs gradient descent algorithm to fine-tune them.1: Two Passes in the Hybrid Learning Procedure for ANFIS 4. ANFIS uses the leastsquares method to identify them. provided that sufficient hidden neurons and fuzzy rules are available. The neurons in ANFIS have different structures. . This approach is thus called hybrid learning method since it combines gradient descent algorithm and least-squares method.

and Output (Algebraic Sum). A sample of ANFIS Editor GUI with input is shown in Figure 4. The output data must be in the last column.52 • The Membership function is defined by parameterized soft trapezoids (Generalized Bell Functions). Any data set is loaded into the ANFIS Editor GUI. open a new Sugeno system or any of the other GUIs to interpret the trained FIS model. . • Functions are linear regressions and multiplication with w . • The Normalization is by Sum and arithmetic division.3 ANFIS editor GUI The ANFIS Editor GUI menu bar can be used to load a FIS training initialization. (or that is applied to the command-line function ANFIS) must be a matrix with the input data arranged as vectors in all but the last column. normalized weights ω. 4. save the trained FIS.5. • The rules are differentiable T-norm usually product. that is.

.6.5: ANFIS Editor GUI 4. This is shown in Figure 4. The sample membership functions shown in the boxes are just icons and do not depict the actual shapes of the membership functions. There is a simple diagram at the top that shows the names of each input variable on the left. and those of each output variable on the right.4 FIS Editor The FIS Editor displays general information about a fuzzy inference system.53 Figure 4.

6: FIS Editor 4. . The Membership Function Editor is the tool that lets you display and edits all of the membership functions associated with all of the input and output variables for the entire fuzzy inference system. Figure 4.54 Figure 4. In fact.7 shows the membership function editor. all of the five basic GUI tools have similar menu options. status lines.5 Membership Functions Editor The Membership Function Editor shares some features with the FIS Editor. and Help and Close buttons.

Rules may be changed. and one connection item as in Figure 4. by clicking the appropriate button.6 Rule Editor The Rule Editor allows user to construct the rule statements automatically. Choosing not under any variable name will negate the associated quality. or added.7: Membership Function Editor 4.55 Figure 4. Choosing none as one of the variable qualities will exclude that variable from a given rule. one item in each output box. by clicking on and selecting one item in each input variable box. . deleted.8.

.56 Figure 4. but.8: Rule Editor 4. for a relatively small number of inputs and outputs. it performs well (depending on how much screen space we devote to it) with up to 30 rules and as many as 6 or 7 variables. Since it plots every part of every rule. The Rule Viewer shows one calculation at a time and in great detail.7 Rule Viewer The Rule Viewer allows users to interpret the entire fuzzy inference process at once. In this sense. it presents a sort of micro view of the fuzzy inference system. it can become unwieldy for particularly large systems. The Rule Viewer also shows how the shape of certain membership functions influences the overall result.

The input is represented by the left-most node and the output by the right-most node. the model structure can be viewed by clicking the Structure button in the middle of the right side of the GUI.9: Rule Viewer 4. Clicking on the nodes indicates information about the structure.57 Figure 4.10. The node represents a normalization factor for the rules. A GUI can be seen in Figure 4. Color coding of branches characterize the rules and indicate whether or not and. or or are used in the rules. not. The branches in this graph are color coded. .8 ANFIS Model Structure After the FIS is generated.

input/output data is collected in a form that will be usable by ANFIS for training. a parameterized model structure (relating inputs to membership functions to rules to outputs to membership functions. This is .10: ANFIS Model Structure 4. Figures 4.12 show the example training error in ANFIS and output of the ANFIS. and so on) is hypothesized. Next. ANFIS can then be used to train the FIS model to emulate the training data presented to it by modifying the membership function parameters according to a chosen error criterion.11 and 4. First.58 Figure 4.9 Modeling Data through ANFIS The modeling approach used by ANFIS is similar to many system identification techniques. In general. this type of modeling works well if the training data presented to ANFIS for training (estimating) membership function parameters is fully representative of the features of the data that the trained FIS intend to model.

data is collected using noisy measurements. however. [Jang. and the training data cannot be representative of all the features of the data that will be presented to the model. Figure 4. In some cases. This is where model validation comes into play. 1993].59 not always the case.11: Training Error .

Another type of data set can also be used for model validation in ANFIS.10 Model Validation (Checking and Testing Data) Model validation is the process by which the input vectors from input/output data sets on which the FIS was not trained. When the checking and training data are presented to ANFIS. This is accomplished with the ANFIS Editor GUI using the so-called testing data set. This other type of validation data set is referred to as the checking data set and it set is used to control the potential for the model over fitting the data. are presented to the trained FIS model. and its use is described in a subsection that follows.60 Figure 4. the FIS model having parameters associated with the minimum checking data model error is then selected.12: Output of ANFIS 4. to see how well the FIS model predicts the corresponding data set output values. .

hopefully this data contains all the necessary representative features. yet sufficiently distinct from the training data set so as not to render the validation process trivial. .61 One problem with model validation for models constructed using adaptive techniques is selecting a data set that is both representative of the data the trained model intends to emulate. so the process of selecting a data set for checking or testing purposes is made easier. If a large amount of data has been collected.

Figure 5.CHAPTER 5 RESULTS AND DISCUSSIONS 5. The training data set was used to train the ANFIS.1 which is in the form of bar chart.1: Training and Checking Data . The partition of data used is shown in Figure 5. whereas the testing data set was used to verify the accuracy and the effectiveness of the trained ANFIS model for the detection of brain tumors.1 Result Various experiments were performed and the sizes of the training and testing sets were determined by taking into consideration the classification accuracies. The data set was divided into two separate data sets – the training data set (20 subjects) and the testing data set (15 subjects).

1.2. There were a total of 27 fuzzy rules in the architecture of the ANFIS using a 3-type (generalized bell.63 The classifier is able to differentiate between normal and abnormal brain.2: ANFIS Errors Type of error Average Training Error 0. The overall result regarding to the ANFIS errors can be seen in Table 5. The ANFIS used 20 training data in 40 training periods and the step size for parameter adaptation had an initial value of 0.1: The ANFIS Errors .0667 Table 5.5x10-3 as shown in Figure 5. triangular and pi) shaped membership function. Figure 5. At the end of 40 training periods. The ANFIS was implemented by using MATLAB software package. the final error convergence value is 2.0038 Checking Error 0.01.1296 Global Error 0.

4 shows the initial and final membership function of inputs 1.3. Therefore. The examination of initial and final membership functions indicates that there are considerable changes in the final membership functions but the change is very small due to less the number of data training. medium. triangular and pi shaped membership function. The outputs of ANFIS are shown in Figure 5. all of the features used in the descriptions of instances may have different levels of relevancy. Membership function of each input parameter was divided into three regions.3: The misclassified output of ANFIS classifier . namely. small. 2 and 3 using the generalized bell. The proposed method gives a framework to detect more brain tumors using ANFIS method for the classification.64 5. Figure 5.2 Performance Analysis In a real world domain. just like the one used in the present study. in the present study changes of the final (after training) membership functions with respect to the initial (before training) membership functions of the input parameters were examined. and large. Figure 5.

65 a) b) Figure 5. .4: a) Membership function before training b)Membership function after training.

33% and 96%. 5. Tables 5.2: The ANFIS Test Results Statistical parameters Specificity Sensitivity Values 93.7. Results from the ANFIS classifier are all displayed as a summary of the performance on the test data.67% Table 5.66 After training. The outputs of ANFIS are shown in Figure 5. 15 testing data was used to validate the accuracy of the ANFIS classifier for the detection of brain tumors. sensitivity and accuracy. . the ANFIS classified MRI and Iris subjects with the accuracy of 93.2. According to Table 5. As seen from that table. An investigation is carried out to see the performance of this classifier when noise increases.5 to 5. 1 subject was classified incorrectly by the ANFIS.6 show the testing data set with 1%.3.33% 96% Accuracy 94.2. The values of these statistical parameters are given in Table 5. 2% and 3% are implemented to the testing data set to look the performance of the ANFIS classifier with noise consideration.4. respectively. 2% and 3% noise respectively.5 and 5.3: The Values of Statistical Parameters 5. The different percentage of noise which is 1%. The test results of the ANFIS are presented in Table 5. The test performance of the ANFIS was determined by the computation of the statistical parameters such as specificity.3 Noise Consideration The training data set in this project is 0% noise implemented. Type of Data MRI Iris Number of subjects 15 Number of subjects classified correctly 14 25 24 Table 5.

5: Testing Data Set with 2% Noise .66 1201.3 2582.37 1831.06 1616.68 1203.84 1191.68 2369.88 875.64 Table 5.03 1600.67 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 1238 1812 934 2404 1584 1177 1751 869 2345 1585 1179 1689 866 2471 1583 1251.88 1679.76 1769.52 1787 887.54 1723.5 878.77 1706.44 1663.82 Table 5.36 2392.83 1189.22 953.4 1673.11 944.74 1849.65 2557.33 2429.7 1657.4: Testing Data Set with 1% Noise 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 1238 1812 934 2404 1584 1177 1751 869 2345 1585 1179 1689 866 2471 1583 1263.76 884.66 2453.

32 1695.04 2416.33 962.52 1215.95 2607.31 1740.49 1213.11 1867.28 1804.5 896.6: Testing Data Set with 3% Noise Figure 5.64 892.5: The misclassified output of ANFIS classifier with 1% noise .99 2477.68 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 1238 1812 934 2404 1584 1177 1751 869 2345 1585 1179 1689 866 2471 1583 1276.09 1632.1 1689.46 Table 5.

69 Figure 5.6: The misclassified output of ANFIS classifier with 2% noise Figure 5.7: The misclassified output of ANFIS classifier with 3% noise .

By using the SPREAD value of 10 the PNN classifier has misclassified 2 out of 15 subjects.4 Comparisons of Result Probabilistic Neural Network (PNN) is used as the border line or benchmark for this project in investigating the classification of data. Results from the PNN classifier are all displayed as a summary of the performance on both training and test data. 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 Input Vector 1238 1239 1812 1813 934 935 2404 2405 1584 1585 1177 1178 1751 1752 869 870 2345 2346 1585 1648 1179 1180 1689 1690 866 867 2471 2533 1583 1642 Table 5.70 Percentage of Noise 1% 2% 3% Number of Subjects 15 15 15 Number of subjects classified correctly 11 8 6 Table 5.7: The ANFIS Test Results with Noise Consideration 5.8: PNN Target Results Target 1 2 3 4 5 1 2 3 4 5 1 5 3 1 5 . Performance from the PNN and ANFIS method is compared and it is summarized in Table 5.9.

71 Methods Application Data Overall Misclassified Performance Rating PNN 15 2 86.9: Comparison between PNN and ANFIS Classification Results 100 86.67% ANFIS 15 1 93.33% Table 5.67 80 60 40 20 0 2 PNN 1 ANFIS Performance Rating 93.8: Chart Comparison .33 Overall Misclassified Figure 5.

1 Conclusions Medical image classification is an important thing in medicine. Fuzzy set theory plays an important role in dealing with uncertainty when making decisions in medical applications. image classification has many forms and uses.CHAPTER 6 CONCLUSIONS AND FUTURE WORKS 6. currently there is no classification strategy that can accommodate all its applications. The proposed technique involves training the ANFIS classifier for symptoms detection. Fuzzy logic enables us to use the uncertainty in the classifier design and consequently increase the credibility of the system output. Unfortunately. Whether it is for diagnostic purposes or practically applied in image guided surgeries. Neuro fuzzy is suitable and feasible to be used as supportive tools for medical problems. neural network and fuzzy logic or known as neuro fuzzy model has been tested and evaluated for classification and detection of MR images. one that performs classification and the other one that detects the tumors from the images. The system that has been designed is used for classifying and detecting tumors in brain MR images. The system involves two major modules. It allows for biological structures to be isolated non-invasively. The combination of both. .

Set up training and testing matrices. or in combination with a least squares type of method. Design a Sugeno FIS appropriate to the classification problem. the following steps need to be performed: 1. 2. Run the ANFIS algorithm on the training data. which provides a measure of how well the fuzzy inference system is modeling the input/output data for a given set of parameters. Hands optimize the FIS. The computation of these parameters (or their adjustment) is facilitated by a gradient vector. Using a given input/output data set. This allows the fuzzy systems to learn from the modeled data. 5. The parameters associated with the membership functions will change through the learning process.73 ANFIS is the fuzzy logic based paradigm that grasps the learning abilities of ANN to enhance the intelligent system’s performance using a priori Knowledge. Once the gradient vector is obtained. in order to compute the membership function parameters that best allow the associated fuzzy inference system to track the given input/output data. given actual input classification data. The training and testing matrices will be composed of inputs and the desired classification corresponding to those inputs. ANFIS constructs a fuzzy inference system (FIS) whose membership function parameters are tuned (adjusted) using either a backpropagation algorithm alone. Test the results using the testing data. These techniques provide a method for the fuzzy modeling procedure to learn information about a data set. any of several optimization routines could be applied to adjust parameters that will reduce some error measure (usually defined by the sum of the squared differences between actual and desired response). 3. This learning method works similarly to that of neural network. To use ANFIS in the classification problem. 4. The technique that has been proposed here has shown a very encouraging level of performance for the problem of tumor classification and detection in brain .

Based on the result in Chapter 5. this project is successful as it met the objectives of the project and successfully developed. As an overall conclusion. This technique is fast in execution. .74 MR images. Furthermore.2 Future Work To the fields of MR images classification. it is very crucial to determine the accuracy and precision of the output of the analysis. He is only able to gather just a few items on the island to bring back with him because his ship is not big. the explorer has a strong feeling that the island has tremendous development potential. There is a need to apply the classification and detection techniques in many fields especially in the medical field whereby the data is large and time consuming to sort or classify manually. efficient in classification and easy in implementation. especially in the medical problems or field such as lung cancer. it can be concluded that this project has achieved the objectives and problem statements as set earlier in Chapter 2. based on those resources he can see. 6. run and optimized the performance of the classification technique. Nevertheless. This makes the approach worth serious consideration for further development as an automatic tool for image classification in medicine. but the explorer is not able to find out about them because his stay on the island is constrained by time. so as to keep the method as generic as possible. breast cancer and etc. There could be many valuable resources on the island. this thesis is like an explorer who has just landed on a big tropical island. Efforts were made to reduce the amount of a priori knowledge used. Experimental result indicates that the technique is workable with an accuracy greater than 90%.

• Build the complete GUI to interface all the process starting from region of interest to ANFIS classification and detection. By doing this. but also some patient information. • Upgrade the system to the application for different size brain medical images slice as shown in Figure 6. The classification would be better performed after the segmentation method and the segmentation results will be used in classification. convergences and determination of premise or consequent parameters. more training data set can be performed. • The use of modified version of ANFIS model instead of Sugeno type ANFIS model.1. For future work the number of brain medical images should be increased and using the real ones.75 This thesis has only explored a small territory in using the neuro fuzzy approach in medical image classification. In such a system. • Get the extra information that could be very useful in detecting some patterns in developing a certain disease. There are many possible directions for future work. Modification can be done in terms of ANFIS architecture. By increasing the number of image. . Figure 6. not only the images are important. which include the following: • In this project the number of medical image is less. the same region of the tumor but in the different slice of brain medical image can be determined.1: Different slice of brain medical image • Perform the segmentation process to detect a tumor and also its size. learning and training algorithm.

etc in order to get an accurate classification of the normal and abnormal brain image.2 gives the block diagram of the technique.76 • Design a combining multiple classifiers that use different features such as texture features. The method can be useful in the diagnostic of diseases if the possibility of a feature is known in the Region of Interest (ROI) of an image.2: Block Diagram of the Neuro Fuzzy Approach . Figure 6. and the images can be classified into category of diseases. tumor shape and size features. A combined feature is more meaningful than the one feature representation alone and can significantly improve the performance. Figure 6.

“Fuzzy logic alternative for analysis in the biomedical sciences.” Artificial Intelligence in Medicine. Aribi. pp. Oweis and Muna J. 32. “Segmentation of magnetic resonance images using a neuro-fuzzy algorithm”. Joseph P. Belal. Man. 1999. Spencer. Hornak.Sunna’. Steimann. 149-165. pp. “A Combined Neuro Fuzzy Approach for Classifying Image Pixels in Medical Applications”.: Foundations of Neural Networks. 305321. 5. vol.77 REFERENCES 1.A.5-6. A. I. F. 3. Roden. 2002.Y. 1996-2008. L.I.56. 24. N.F. Kuncheva. 121-128.Benamrane. 2006. Journal of Electrical Engineering. MIT Press. “ANFIS: Adaptive-network-based fuzzy inference system. “Fuzzy diagnosis. 8. and Cybernetics.” Artificial Intelligence in Medicine. “Fuzzy Neural Networks and Genetic Algorithms for Medical Images Interpretation”. Kasabov. A.” IEEE Transactions on Systems. Virant-Klun. 1999. N. S. S. Cambridge . Taktak. J. pp. IEEE Proceedings of the Geometric Modeling and Imaging. 6. .” Computers and Biomedical Research. 23(3). 7. 2. Virant. S. The Basics of MRI. 16. Ramiro Castellanos and Sunanda Mitra. 9. vol.Kraoula. 4. A. Nevill. pp. 2000. vol.J. 2005.1996. 665685. Bevan. 1993. L. D. No. Vol. Jang. Fuzzy Systems and Knowledge Engineering. J. “Automatic detection of distorted plethysmogram pulses in neonates and paediatric patients using an adaptive-network-based fuzzy inference system.G. IEEE Symposium on Computer-Based Medical Systems.-S.R. Rami J. vol.

Inc.Frattale Mascioli.. A. p. ANFIS synthesis by hyperplane clustering. 2005. Panella. "PCA-based feature extraction using class information". J. Fuzzy model identification based on cluster estimation. and E. Kung. Proceedings of the Fifth IEEE International Conference. 12. Journal of Biomedical Engineering. S.355-362. 19. Artificial Intelligence in Medicine. G. Myoung Soo Park. Y. et al. Han Witjes et al. Martinelli. R. Rizzi. 31:73–89. pp. I.-T.S. Lukas L.1: 340 -345. USA. Neural Network Segmentation of Magnetic Resonance Spin Echo Images of the Brain. 20. 1994. Jang. 17. Jin Hee Na. Caramella. Jin Young Choi.M. 16. Cagnoni. 2001. 1996.Valli. 18.9.341-345.. 14. Journal of Intelligent and Fuzzy Systems. “A Novel and Efficient Neuro-Fuzzy Classifier for Medical Diagnosis”. Man and Cybernetics.Vol. Jang JSR.Chiu. 1996. 2003.Fuzzy and Soft Computing. 239-240. Rucci.78 10. Jang.M. Multispectral magnetic resonance image analysis using principal component and linear discriminant analysis. 17:261–269. V 15. IFSA World Congress and 20th NAFIPS International Conference. Chin-Ming Hong et al.Martinelli. V012: 1493 -1499. pp.. Saviello y Tagliaferri. vol. F. Pearson Education Inc. 2006.-S. Fuzzy Systems.. G. Neuro. Sun C T and Muzutani E. Mascioli. Mizutani . 22. 15. A constructive approach to neurofuzzy networks. J. . M. IEEE International Conference on Systems. Sun. 2004. Neuro-Fuzzy and Soft Computing-A computational approach to learning and machine intelligence. C. Brain tumor classification based on long echo proton MRS signals. 64 : 347-358. no 5 September 1998. 1997. Diamantaras and S. 1993. 2007. IEEE International Joint Conference on Neural Networks. Journal of Magnetic Resonance Imaging.R. K. 2 (3): 267-278. Meneganti. 13.L. IEEE Transactions on Neural Networks.F. New Jersey: Prentice Hall. John Wiley & Sons. “Principal Component Neural Networks: Theory and Applications”. 11... Fuzzy Neural Network for Classification and Detection of Anomalies. Coppini. 1998. Input selection for ANFIS learning. Signal Processing. F. 21.

Image Processing Toolbox. 1992. Wiley. Baltimore. vol. Attikiouzel. 27. 1998. L.” IEEE Bans. 101-108. Mukaidono and I . Neural Networks and Prior Knowledge Help the Segmentation of Medical Images. Hall. 32.Benamrane and al. pp. Morrison and Y. Fuzzy toolbox. 3. Digital Image Processing. Turksen . Neural Nets. Conf. M. 26. 892-896. “ A fuzzy neural network for pattern classification and feature selection . Anil Jain. Vol. C. 25.6 No.mathworks. Rafael. . 2002. 28. E. Richard. Paul Suetens. England. R. pp. M. 3.1989. Addison Wesley.. “A probabilistic neural network based image segmentation network for magnetic resonance images. Nauck. F. R. Cambridge University Press. Matlab Home. p. Massachusetts. 29. 1992. 2(4).79 23. Prentice Hall. 130. pp. 31. Chichester. Fundamentals of Medical Imaging. “A Hybrid Fuzzy Neural Networks for the Detection of Tumors in Medical Images”. 33. 2005. 0. Journal of Computing and Information Technology. User manual. 1999. 24. ” Fuzzy Sets Sys. N. Neural Nets. et al. P.. Klawonn. Mathworks Inc. American Journal of Applied Sciences. Poli. vol.. Woods.. 30. B .” Proc.. “A comparison of neural network and fuzzy clustering techniques in segmenting magnetic resonance images of the brain. London. Valli.. Math works worldwide. and Kruse. analysis and algorithm development: http://www. 2002. Fundamentals of Digital Image Processing. Cagnoni and Coppini.. Li. Gonzalez. Image Processing. Foundations of Neuro-Fuzzy Systems. D. 1997. 60-65.. 672-682. Vol.

%find differential between image a and b d = c>20. %find the point bigger than 20 fprintf('\n Location of abnormal tissues. %Resize an image to 64x64 pixel a3 = rgb2gray(a2).j]=find(d). %Convert an image to a binary image Ifill = imfill(Ibw.bwarea(Ifill).'hole').. Ibw = im2bw(Igray. a2 = imresize(al. al = imread(imagname1). %convert an RGB image or colormap to grayscale imagname2 = input('\n Enter the name of the input image file (filename..80 APPENDIX A MATLAB source code for Transformation image to Data %Entering the image for MATLAB fprintf('\n Entering the image for MATLAB. %Convert an image to a binary image I2fill = imfill(I2bw.') [i. %Convert an RGB image or colormap to grayscale %Test size of priori model and input model Igray = rgb2gray(al).title('abnormal') X = bwarea(I2fill) .graythresh(I2gray)).title('original') I2gray = rgb2gray(b1).'s'). %Resize an image to 64x64 pixel b3 = rgb2gray(b2).graythresh(Igray)).[64 64]). %Fills holes in the binary image BW figure (1).ext):'.'hole'). I2bw = im2bw(I2gray. b2 = imresize(b1.. %Fills holes in the binary image BW figure (2). %location point bigger than 20 .[64 64]). %substract two binary image if X <=10 %if size of input binary image match with prior image c = a3-b3. b1 = imread(imagname2).imshow(Ifill)..') imagname1 = input('\n Enter the name of the prior image file (filename.ext):'.'s').imshow(I2fill).

end fprintf('\n Coordinate of damage tissues:')..'s').81 % Find the coordinate T=[ ].name) else fprintf('\n You choose NOT to SAVE the Result') end . end end T=[T L(k)].. B=B+1. T %Saving the output sav = input('\n Do you like to SAVE Result? (y/n) : '. for A=j(k) while A~=B L(k)=i(k)+C.'ans'). fprintf('\n Saving. for k=1:3 B=0.') save(name. fprintf('\n The new file is called %s and it is saved in MATLAB working Directory'. C=0. C=C+64. if (sav == 'y') fprintf('\n You choose to SAVE the Result') name = input('\n Type the name of the new file (filename) : '.'s').

82 %If size of input Dot match with prior image else fprint('\n Size of input image not match with size of prior image, Please Try again!!') return end



PCA Algorithm code

% PCA-based brain mri recognition system


clear all clc

% Initial directory paths can be customized and fixed TrainDatabasePath = uigetdir('D:\Program Files\MATLAB\R2006a\work', 'Select training database path' ); TestDatabasePath = uigetdir('D:\Program Files\MATLAB\R2006a\work', 'Select test database path');

prompt = {'Enter test image name (a number between 1 to 10):'}; dlg_title = 'Input of PCA-Based Brain MRI Recognition System'; num_lines= 1; def = {'1'};

TestImage = inputdlg(prompt,dlg_title,num_lines,def); TestImage = strcat(TestDatabasePath,'\',char(TestImage),'.jpg'); im = imread(TestImage);

T = CreateDatabase(TrainDatabasePath); [m, A, EigenMRI] = EigenMRICore(T); OutputName = TumorRecognition(TestImage, m, A, EigenMRI);

SelectedImage = strcat(TrainDatabasePath,'\',OutputName);

84 SelectedImage = imread(SelectedImage);

imshow(im) title('Test Image'); figure,imshow(SelectedImage); title('Equivalent Image');

str = strcat('Matched image is : ',OutputName); disp(str)

containing all 1D image vectors. % % % Argument: % % Returns: % % % % T . . Then.A 2D matrix. % Number of all images in the training database end end %Construction of 2D matrix from 1D image vectors T = [ ].Path of the training database %File management TrainFiles = dir(TrainDatabasePath). Train_Number = 0. it puts these 1D column vectors in a row to % construct 2D matrix 'T'. Suppose all P images in the training database have the same size of MxN.'.')|strcmp (TrainFiles(i). for i = 1:size(TrainFiles.1) if not(strcmp(TrainFiles(i).. TrainDatabasePath . So the length of 1D column vectors is MN and 'T' will be a MNxP 2D'.')|strcmp(TrainFiles(i).'Thumbs.db')) Train_Number = Train_Number + 1.85 APPENDIX C CreateDatabase function code function T = CreateDatabase(TrainDatabasePath) % Description: This function reshapes all 2D images of the training database % into 1D column

str = strcat(TrainDatabasePath. str = strcat('\'.jpg').1).str). % 'T' grows after each turn End . it is not mandatory! str = int2str(i).'. [irow icol] = size(img). img = rgb2gray(img). However. temp = reshape(img'.str.86 for i = 1 : Train_Number % Chose the name of each image in databases as a corresponding number. % Reshaping 2D images into 1D image vectors T = [T temp].irow*icol. img = imread(str).

m.i)) . So the length of 1D column vectors is M*N and 'T' will be a MNxP 2D matrix.(M*Nx1) Mean of the training database T .(M*Nx(P-1)) Eigen vectors of the covariance matrix of the training database . EigenMRI .2).2).(M*NxP) Matrix of centered image vectors %Calculating the deviation of each image from mean image A = [ ]. A m . % Computing the average MRI image m = (1/P)*sum(Tj's) Train_Number = size(T. for i = 1 : Train_Number temp = double(T(:. Suppose all P images in the training database have the same size of MxN.A 2D matrix. EigenMRI] = EigenMRICore(T) % Description: This function gets a 2D matrix. containing all training % image vectors and returns 3 outputs which are extracted from training % database. containing all 1D image vectors.87 APPENDIX D EigenMRICore function code function [m. % % Argument: % % % % % % Returns: % % % % %Calculating the mean image m = mean(T. A. A = [A temp]. % Merging all centered images end .

88 %Snapshot method of EigenMRI methods L = A'*A. for i = 1 : size(V. [V D] = eig(L).i)]. end end %Calculating the eigenvectors of covariance matrix 'C' EigenMRI = A * L_eig_vec. % Diagonal elements of D are the eigenvalues % for both L=A'*A and C=A*A'. %Sorting and eliminating eigenvalues L_eig_vec = [ ].2) if( D(i.i)>1 ) L_eig_vec = [L_eig_vec V(:. % L is the surrogate of covariance matrix C=A*A'. % A: centered image vectors .

(M*Nx(P-1)) Eigen vectors of the covariance matrix of the training database. m.(M*NxP) Matrix of centered image vectors. % % % A .Name of the recognized image in the training database. % % Returns: OutputName .89 APPENDIX E TumorRecognition function code function OutputName = TumorRecognition(TestImage.Path of the input test image %Projecting centered image vectors into imagespace ProjectedImages = [ ]. EigenMRI) % Description: This function compares two brain images by % projecting the images into imagespace and % measuring the Euclidean distance between them. which is output of 'EigenMRICore' function. for i = 1 : Train_Number temp = EigenMRI'*A(:. which is output of 'EigenMRICore' function. % Projection of centered images into imagespace . which is output of 'EigenMRICore' function. TestImage . % % % % EigenMRI .(M*Nx1) Mean of the training database.2). A.i). Train_Number = size(EigenMRI. % % Argument: % % % m .

irow*icol. end [Euc_dist_min. InImage = reshape(temp'.'. end %Extracting the PCA features from test image InputImage = imread(TestImage). .i). for i = 1 : Train_Number q = ProjectedImages(:. [irow icol] = size(temp). % Test image feature vector %Calculating Euclidean distances Euc_dist = [ ].90 ProjectedImages = [ProjectedImages temp].Recognized_index] = min(Euc_dist).1).:. Difference = double(InImage)-m.jpg'). temp = ( norm( ProjectedTestImage . OutputName = strcat(int2str(Recognized_index).1). Euc_dist = [Euc_dist temp].q ) )^2. % Centered test image ProjectedTestImage = EigenMRI'*Difference. temp = InputImage(:.

91 APPENDIX F Neuro Fuzzy Algorithm code % Classification using ANFIS % Input Phase clear name_trainingdata=input('\nEnter name of Training Data files (only prefix. without neither number nor extension): '. loadingchecking=load(axus1). bar(loadingchecking). axus1=strcat(name_checkingdata.'s'). subplot(212). data=[loadingtraining loadingoutput]. title('Training data').'s'). name_outputdata=input('\nEnter name of Output file (only prefix. name_checkingdata=input('\nEnter name of Checking Data file (only prefix. figure(1). % strcat combine two strings together axus=strcat(name_trainingdata.'trimf'. numMFs=input('\nNumber of membership function: '). % Loading Training data loadingtraining=load(axus).'s').extension). subplot(211).extension).dat'.'pimf'). epoch=input('\nNumber of epoch for training process: '). extension='. bar(data). without neither number nor extension): '. axus2=strcat(name_outputdata. outmftype=('linear').extension). without neither number nor extension): '. . loadingoutput=load(axus2). inmftype=str2mat('gbellmf'.

'input2'. time=toc. subplot(312).errorchecking]=anfis(data.numMFs.errortraining. plotmf(fis.stepssize.1) . plotmf(fistraining. plotmf(fistraining. % Classification/Fuzzy Inference System Phase to get the FIS fis=genfis1(data.3).3).checkfis. figure(3) subplot(311).'input'. [fistraining.[1 1 1 1]. legend('input1'.epoch.92 title('Checking data'). subplot(312). % Anfis Model Training Phase % Recording time for ANFIS training tic.loadingchecking). plotmf(fis.2).fis. subplot(311).inmftype.'output').outmftype) figure(2).2).'input'. plotmf(fis. Total_time_for_ANFIS_training=time fprintf('\n<<<<<<<<<<<<<<<<<<<<WARNING!!!!>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>') fprintf('\nResults above show second column is training error') fprintf('\nand third column is checking error') fprintf('\n<<<<<<<<<<<<<<<<<<<<WARNING!!!!>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>') fprintf('\n') % Display Error for Training and Checking final_training_error=errortraining(epoch. subplot(313).'input'.'input'.'input3'.'input'.1).'input'.1). subplot(313). plotmf(fistraining.

extension).extension). end for i=1:size(output) . expected_result=input('\nEnter name of Expected Data Result files (only prefix. if saving==1. [a.d]=size(target). loading=load(applicationdata).b]=size(output). out=evalfis(loading. [c.without neither number nor extension): '. minimum_training_error=min(errortraining). output=round(out) % Detection Phase target=load(axus3). % Application Phase %Evaluate the fis based on new set of data(application data) name_applicationdata=input('\nEnter name of Application Data files (only prefix.'s'). writefis(fistraining).93 final_checking_error=errorchecking(epoch. eg: yes=1 or no=0?= ').fistraining). minimum_checking_error=min(errorchecking).without neither number nor extension): '. applicationdata=strcat(name_applicationdata. average_training_error=(maximum_training_error+minimum_training_error)/2 maximum_checking_error=max(errorchecking). else saving==2.'s'). if a~=c error('Mismatched input and output data matrices'). number=0. axus3=strcat(expected_result. end.1) maximum_training_error=max(errortraining). average_checking_error=(maximum_checking_error+minimum_checking_error)/2 average_global_error=(average_training_error+average_checking_error)/2 % Save the trained FIS saving=input('\nSave the Trained FIS.

1)~=target(i.1). disp('Correct') elseif output(i. end end Total_Misclassified=number .1).1)==target(i.94 if output(i. disp('InCorrect') number=number+1.

95 APPENDIX G Brain MR Image for Class 1 MRI1-1 MRI1-2 MRI1-3 MRI1-4 .

96 APPENDIX H Brain MR Image for Class 2 MRI2-1 MRI2-2 MRI2-3 MRI2-4 .

97 APPENDIX I Brain MR Image for Class 3 MRI3-1 MRI3-2 MRI3-3 MRI3-4 .

98 APPENDIX J Brain MR Image for Class 4 MRI4-1 MRI4-2 MRI4-3 MRI4-4 .

99 APPENDIX K Brain MR Image for Class 5 MRI5-1 MRI5-2 MRI5-3 MRI5-4 .

Sign up to vote on this title
UsefulNot useful