MEDICAL IMAGE CLASSIFICATION AND SYMPTOMS DETECTION USING NEURO FUZZY

MOHD ARIFFANAN BIN MOHD BASRI

A project report submitted in partial fulfillment of the requirements for the award of the degree of Master of Engineering (Electrical – Mechatronics and Automatic Control)

Faculty of Electrical Engineering Universiti Teknologi Malaysia

NOVEMBER 2008

iii

Specially..

To my beloved mother and late father, My kind brothers and sisters And not forgetting to all my friends For their Love, Sacrifice, Encouragement, and Best Prayers

iv

ACKNOWLEDGEMENTS

Praise be to Allah S.W.T to Whom we seek help and guidance and under His benevolence we exist and without His help this project could not have been accomplished.

I would like to express my sincere thanks and appreciation to Dr. Hj. Mohd. Fauzi bin Othman for his supervision, advice, guidance, encouragement and insightful comments given throughout the course of completing this project.

I am also indebted to the Universiti Teknologi Malaysia for their encouragement and financial support during my study. Not forgetting to all my fellow postgraduate students and friends for their moral support and given throughout the entire masters programme. Without their continued support and interest, this project would not have been realized.

Last but not least, my gratitude also goes to all my family members for their continuous encouragement and support. Thank you all.

Operator-assisted classification methods are impractical for large amounts of data and are also non-reproducible. The performance of the ANFIS classifier was evaluated in terms of training performance and classification accuracies and the results confirmed that the proposed ANFIS classifier has potential in detecting the tumors. for instance. The use of artificial intelligent techniques.v ABSTRACT The conventional method in medicine for brain MR images classification and tumor detection is by human inspection. Hence. in this project the neuro fuzzy system or ANFIS was applied for classification and detection purposes. MR images also always contain a noise caused by operator performance which can lead to serious inaccuracies classification. neural networks. Decision making was performed in two stages: feature extraction using the principal component analysis (PCA) and the ANFIS trained with the backpropagation gradient descent method in combination with the least squares method. neuro fuzzy have shown great potential in this field. . fuzzy logic.

Prestasi pengkelas ANFIS berkenaan dengan prestasi latihan dan ketepatan pengkelasan telah diuji dan hasilnya telah menunjukkan bahawa pengkelas ANFIS yang dicadangkan mempunyai potensi untuk mengesan barah. Teknik pengkelasan yang dibantu oleh operator adalah tidak praktikal untuk jumlah data yang besar dan juga tidak dapat dihasilkan. fuzzy logic dan juga neuro fuzzy telah menunjukkan potensi yang besar di dalam bidang ini.vi ABSTRAK Teknik konvensional di dalam perubatan untuk pengkelasan dan pengesanan barah untuk imej magnetik resonan otak adalah dilakukan secara pemeriksaan manual. di dalam projek ini sistem neuro fuzzy atau juga dikenali sebagai ANFIS telah diaplikasikan untuk tujuan pengkelasan dan pengesanan. Pengambilan keputusan telah dilakukan secara dua peringkat: pengekstrakan sifat menggunakan prinsip analisis komponen dan ANFIS terlatih dengan penggabungan teknik ‘backpropagation gradient descent’ dan juga ‘least squares’. neural network. Oleh yang demikian. Penggunaan teknik kecerdikan buatan contohnya. . Imej magnetic resonan juga lazimnya mengandungi hingar yang berpunca daripada mutu kerja operator yang mana boleh menjurus kepada ketidaktepatan pengkelasan yang serius.

2 Learning from Biology Generic Model of Neural Network 8 8 11 11 13 .vii TABLE OF CONTENTS CHAPTER TITLE PAGE DECLARATION DEDICATION ACKNOWLEDGMENTS ABSTRACT ABSTRAK TABLE OF CONTENTS LIST OF TABLES LIST OF FIGURES LIST OF ABBREVIATIONS LIST OF SYMBOLS LIST OF APPENDICES ii iii iv v vi vii x xi xiii xiv xv 1 INTRODUCTION 1.5 1.3 1.2.2.6 Project Background Magnetic Resonance Imaging Problem Statement Objectives Scope of Work and Methodology Thesis Outline 1 1 2 5 5 6 7 2 LITERATURE REVIEW 2.1 1.2 Related Works Artificial Neural Networks 2.2 1.1 2.4 1.1 2.

3 2.8 Data Modeling System developing 3.7 3.3 3.8.viii 2.2.8.1 3.4 2.2 3.3.3.3 Bias and Threshold Activation Functions Network Learning Phase 14 15 15 16 17 18 19 20 21 21 Fuzzy Logic Systems 2.4 Principal Component Analysis (PCA) 3 RESEARCH METHODOLOGY 3.2 3.8.5 2.2 2.2.1 2.3 2.3.5 2.4 2.6 Data Management and Preparation Data Mining Input Phase FIS Phase ANFIS modeling phase Output phase 27 27 27 28 30 31 32 34 37 37 38 38 39 40 42 .1 3.2.5 3.8.6 2.3.6 Methodology Proposed Method Transformation of Image to Data Form Pixels Labeling Region of Interest Extraction Feature Extraction using Principal Component Analysis (PCA) 3.6 Advantages of Fuzzy Logic Disadvantages of Fuzzy Logic Fuzzy Sets Membership functions Components of a fuzzy system Fuzzy Logic Graphical User Interface (GUI) 24 25 2.2.3.8.3.5 3.4 3.8.4 3.3 3.

2 5.5 4.3 4.2 4.2.99 .7 4.2 Conclusions Future Works 72 72 74 REFERENCES Appendices A-K 77 80 .4 Result Performance Analysis Noise Consideration Comparisons of Result 62 62 64 66 70 6 CONCLUSIONS AND FUTURE WORKS 6.6 4.1 4.10 ANFIS Architecture Learning algorithm of ANFIS ANFIS Classifier 43 43 44 45 49 51 52 53 54 55 56 57 58 ANFIS editor GUI FIS Editor Membership Functions Editor Rule Editor Rule Viewer ANFIS Model Structure Modeling Data through ANFIS Model Validation (Checking and Testing Data) 60 5 RESULTS AND DISCUSSIONS 5.3 4.1 6.2 Neuro Fuzzy Model Adaptive neuro fuzzy inference system (ANFIS) 4.2.3 5.1 4.8 4.2.1 5.4 4.9 4.ix 4 NEURO FUZZY MODELING AND SOFTWARE IMPLEMENTATION 4.

4 5.9 TITLE Two passes in the hybrid learning procedure for ANFIS The ANFIS errors The ANFIS test results The values of statistical parameters Testing data set with 1% noise Testing data set with 2% noise Testing data set with 3% noise The ANFIS test results with noise consideration PNN target results Comparison between PNN and ANFIS PAGE 51 63 66 66 67 67 68 70 70 71 . 4.2 5.5 5.x LIST OF TABLES TABLE NO.8 5.1 5.6 5.3 5.1 5.7 5.

6 2.12 3.4 3.6 3.3 2.xi LIST OF FIGURES FIGURE NO.1 MRI machine TITLE PAGE 3 4 4 13 14 16 21 21 22 24 28 30 30 31 32 34 36 37 39 40 41 41 42 46 Schematic diagram of MRI machine Brain MRI Biology neuron Generic model of artificial neural network Feedforward Network Membership functions of a crisp set and a fuzzy set Membership grades of a fuzzy set Components of a fuzzy logic system Fuzzy Logic Graphical User Interface (GUI) The proposed system Convert brain image to binary image Pixel coordinate system Advanced coordinate system for matrix 16 by 16 Location of tumor Schematic diagram of a MR image recognizer Training Data Set Basic flow diagram of computations in ANFIS Partial of input phase FIS generated thru genfis1 Information of the ANFIS model ANFIS training errors and total time training Sample of classification results A two-input first-order Sugeno fuzzy model with two rules .9 3.4 2.2 2.3 3.5 2.1 3.7 3.10 3.11 3.1 2.2 1.13 4.8 3.7 3.1 1.5 3. 1.2 3.3 2.

5 5.7 4.6 4.6 5.8 6.12 5.10 4. Two-input first-order Sugeno fuzzy model with nine rules The input space that are partitioned into nine fuzzy regions ANFIS editor GUI FIS editor Membership function editor Rule editor Rule viewer ANFIS model structure Training error Output of ANFIS Training and checking data ANFIS errors The misclassified output of ANFIS classifier a)Membership function before training b)Membership function after training The misclassified output of ANFIS classifier with 1% noise The misclassified output of ANFIS classifier with 2% noise The misclassified output of ANFIS classifier with 3% noise Chart comparison Different slice of brain medical image Block Diagram of the neuro fuzzy approach 46 49 49 53 54 55 56 57 58 59 60 62 63 64 65 68 69 69 71 75 76 .5 4.xii 4.1 6.4 5.4 4.2 ANFIS architecture.3 4.2 5.9 4.3 5.8 4.11 4.2 4.1 5.7 5.

xiii LIST OF ABBREVIATIONS AFLC ANFIS ANN CAT COG COS CT FCM FIS GUI MF MLP MR MRI NMR NN PCA PET PNN RF RGB ROI - Adaptive Fuzzy Leader Clustering Adaptive Neuro Fuzzy Inference System Artificial Neural Network Computed Axial Tomography Centre of Gravity Centre of Singleton Computed Tomography Fuzzy C-Means Fuzzy Inference System Graphical User Interface Membership Function Multi Layer Perceptron Magnetic Resonance Magnetic Resonance Image Nuclear Magnetic Resonance Neural Network Principal Component Analysis Positron Emission Tomography Probabilistic Neural Network Radio Frequency Red Green Blue Region of Interest .

xiv LIST OF SYMBOLS θ i w b O ∑ Π x e E ∈ µA(x) ∩ ∪ - threshold Input signals Weight the neurons Bias signal Output signals Summation of all incoming signals Product of all the incoming signals Universal sets error total error Subset Membership functions of A Intersection @ AND logic operation Complement @ OR logic operation activation function Rows x columns training image of image i pixel vector feature vectors f MxN i Φ ωi .

xv LIST OF APPENDICES APPENDIX A B C D E F G H I J K TITLE MATLAB source code for transformation image to data PCA Algorithm code CreateDatabase function code EigenMRICore function code TumorRecognition function code Neuro Fuzzy Algorithm code Brain MR Image for Class 1 Brain MR Image for Class 2 Brain MR Image for Class 3 Brain MR Image for Class 4 Brain MR Image for Class 5 PAGE 80 83 85 87 89 91 95 96 97 98 99 .

The idea is to reduce human error as much as possible by assisting . The problem is that it is not very easy to obtain such results. It has been proven that double reading of medical images could lead to better tumor detection. therefore good software to assist humans in medical institutions is of great interest nowadays. the accuracy must be very high and the false negative rate must be low.CHAPTER 1 INTRODUCTION 1. Also. With all this effort done in the research field. different types of X-rays. there is still a lot of place for improvements and the medical image processing is a domain in continuous expansion. such as scans. computer assistance is highly sought in medical institutions due to the fact that it could improve the results of humans in such a domain where the false negative cases must be at a very low rate. Different approaches that can produce medical images must be studied. MRI images and other radiological techniques. Also.1 Project Background Automated classification and detection of tumors in different medical images demands high accuracy since it deals with human life. Why is this domain in continues expansion and without and good accepted method? This is due to the fact that in such an important domain. A lot of methods have been proposed in the literature for CT (Computed Tomography). But the cost incurred in double reading is very high. the technique that produces those images is very important in order to know what to apply to a certain medical image in order to get better results.

The technique is called magnetic resonance imaging rather than nuclear magnetic resonance imaging because of the public's negative associations with the word nuclear in the late 1970's. Neuro fuzzy systems are fuzzy systems which use ANNs theory in order to determine their properties (fuzzy sets and fuzzy rules) by processing data samples. therefore automatic and reliable methods for tumor detection are of great need and interest. A specific approach in neuro fuzzy development is the adaptive neuro fuzzy inference system (ANFIS). In this project the automated classification of brain magnetic resonance images by using some prior knowledge like pixel intensity and some anatomical features is proposed. 1. Successful implementations of ANFIS in biomedical engineering have been reported. Since currently there are no widely accepted methods. is primarily a tomography imaging technique . classification or clustering techniques is essential to the developments of neuro fuzzy systems particularly in medical-related problems. including the brain. This is important since it involve saving human lives. like other imaging techniques such as CAT and PET. which is used by chemists to study the properties of molecules. MRI is a type of nuclear magnetic resonance (NMR). fully understanding the detection. which has shown significant results in modelling nonlinear functions.2 Magnetic Resonance Imaging Magnetic resonance imaging (MRI) is a procedure used to produce high resolution images of the inside of the body.2 physicians and radiologists with some software that could lead to better results. fuzzy set theory plays an important role in dealing with uncertainty when making decisions in medical applications [1]. Furthermore. MRI. These include the clustering and classification techniques especially for MR images problems with huge scale of data which consumes time and energy if done manually. for classification [3] and data analysis [4]. The application of neuro fuzzy systems in the classification and detection of data for MR images problems are not fully utilized yet. The ANFIS learns features in the data set and adjusts the system parameters according to a given error criterion [2]. Thus.

This form of imaging is. Figure 1. to the untrained eye. the MRI is formed by an MRI camera or scanner and the slices are composed of groups of picture elements or pixels displayed on a computer monitor. Figure 1. This very strong magnetic field is then used to align the hydrogen nuclei of the tissue to be imaged. very much like a real anatomical slice through the body. Also placed near the tissue to be imaged is another coil of wire called an RF coil (RF because it generates/reads radio frequency fields). In order to generate a magnetic field of this strength the coil is cooled to near absolute zero with liquid helium. However. In MRI the subject or patient is placed inside the scanner which is made up of a moveable bed-like structure and a large hollow tube. The RF coil is used to both change the energy state of the hydrogen nuclei and to record the RF output of these perturbations. in some respects. an MRI image may be indistinguishable from a slice taken from a post-mortem body.1 shows a picture of an MRI machine and Figure 1. unlike a post-mortem slice. transformed and then displayed [5].3 ("tomo" means slice).1: MRI machine . This means that it produces an image in a thin slice through the body.2 shows the schematic of an MRI machine.5 and 4 Tesla) which is strong enough to accelerate a paper clip to near lethal velocities across the distance of a small room. The latter are the raw data of an MRI which are recorded by the computer. The tube contains a thick coil of wire that generates a very intense magnetic field (between 1. In fact.

Figure 1.3: Brain MRI .3.4 Figure 1.2: Schematic diagram of MRI machine In MRI. water molecules give off radio signals which are converted into high resolution images that look like a picture shown in figure 1.

4 Objectives The main objective of this project is to propose an efficient classification technique for magnetic resonance (MR) images using neuro fuzzy. neuro fuzzy has shown great potential in this field. complex and cognitive process. This includes the neuro fuzzy architecture and the adaptation algorithm for the neuro fuzzy structure itself. The MR images data is by nature. classification and detection of algorithms which have direct applications in many medical problems have become much easier to be implemented and diagnosed.5 1. fuzzy logic.3 Problem Statement The conventional method in medicine for brain MR images classification and tumor detection is human inspection. The neuro fuzzy models hereby refer to the ANFIS (Adaptive Neuro Fuzzy Inference System). the study on the fundamental theory of neuro fuzzy system have to be done. With the involvement of soft computing. Accurate diagnosis of MR images data is not an easy task and is always time consuming. Besides. MR images also always contain a noise caused by operator performance which can lead to serious inaccuracies classification. Hence. 1. a sample medical image will be tested on the model to observe the classification and detection performance and subsequently optimization . this project tries to find out how the neuro fuzzy system or the hybrid combination of neural network and fuzzy logic techniques can be applied in the classification and detection purposes. diagnosis with wrong result and delay in delivery of a correct diagnosis decision could occur due to the complexity and cognitive process of which it is involved. Operator-assisted classification methods are impractical for large amounts of data and are also non-reproducible. neural networks. particularly on the medical field. In some extreme scenario. The use of artificial intelligent techniques for instance. the pattern matching. a huge. Hence.

Deal only with the same size of the brain medical images slice. . testing and tuning or optimization parameters.6 of the chosen technique will be carried out.5 Scope of Work and Methodology The scope of the project is as follows:i. Distinguish between normal and abnormal of the brain medical images. ii. hopefully an automated system that provides high accuracy of brain medical images classification and symptoms detection that probably can replace the current human visual inspection methods is achieved. Classify the abnormal brain medical images based on the region of abnormal tissues using neuro fuzzy. iii. v. adaptation and training of the neuron using the MATLAB. ix. Training. Extract the region of abnormal tissues using image processing tool in MATLAB. Process the brain medical images using PCA. This project starts with converting the brain medical images into a form of data using MATLAB. simulation. iv. The brain medical images also have to be pre-processed using PCA to reduce the dimensionality of the data. The neuro fuzzy model considered hereby refers to ANFIS (Adaptive Neuro Fuzzy Inference System). vi. Develop the neuro fuzzy algorithm for classification using MATLAB. ii. The methodologies of project are as follows: i. Involves the neuro fuzzy model coding. vii. Analyzing the neuro fuzzy model performance accuracy. Convert the brain medical images into a form of data using MATLAB. As the ultimate goal of this project. 1. iv. viii. iii.

some overview including the background. problem statement. the conclusion of this project and some recommendations for future work are discussed in Chapter 6. Lastly. optimize the performance of the proposed technique.7 v. . objectives. vi.6 Thesis Outline This thesis is organized as follows. In Chapter 1. scope of work and methodology are presented. the extraction of a region of interest. related works and also theories regarding this project are included in Chapter 2. Then. Finally. this proposed technique will be coded in MATLAB and subsequently to be trained and tested with the brain medical images data. Some literature reviews. transformation of image into a form of data and also data and ANFIS modelling. Chapter 3 contains a research methodology of the proposed method. The neuro fuzzy architecture and software development and implementation for classification are discussed in Chapter 4. 1. Chapter 5 presents the result from the simulation and discusses the performance analysis.

Rami J. The method showed high quality classification for images of simple components. where it uses neural network’s ability to learn. (2006) has proposed an approach for detection and specification of anomalies present in medical images.Benamrane et al.Oweis et al. Each pixel is assigned a degree of membership for each of the three fuzzy subsets.CHAPTER 2 LITERATURE REVIEW 2. the Fuzzy Neural . The Genetic Algorithm adds to this hybridizing the property of total research like an initialization of the Fuzzy Neural Networks training algorithm which is based on an adapted version of the back propagation algorithm... Fuzzy Logic and Genetic Algorithms in a hybrid system.1 Related Works This section will present some approaches that have been developed and used in MRI images. The idea is to combine three metaphors: Neural Networks. After applying the growing region algorithm to extract regions. respectively. A training set is used to create and train the classifier system. (2005) present the pixel classification of biomedical image using neuro fuzzy approach. The method is based on the spatial properties of the image features and makes use of multi-scaled representations of the image. The Neural Networks and Fuzzy Logic metaphors are coupled in one system called Fuzzy Neural Networks. The classes are represented as fuzzy sets with degrees of memberships. Classified pixels are finally shown as three separate images each representing a set. N. and membership degrees and functions of fuzzy logic.

The inputs to the MLP are limited to normalized intensity values of the pixels from a (7 × 7) window scanned across the image.. where the value of a vigilance parameter restricts the number of prototypes representing the feature vectors. (2003) reports on the combined use of a neural network(a multi layer perceptron. This study proposes a refined K-means clustering algorithm and a gradient-based learning rule to logically determine and adaptively tune the fuzzy . Training sets are first developed using a lengthy iterative process. Ian Middleton et al. This approach performs vector quantization by updating the winning prototype of a competitive network through an unsupervised learning process. Thereafter. as well as faster. non-iterative selection of training examples... The use of spatial coordinates as additional inputs to the MLP is then shown to provide an improvement in segmentation performance as quantified using the effectiveness measure (a weighted product of precision and recall). Subsequently. MLP) and active contour model (‘snake’) to segment structures in magnetic resonance (MR) images. (2000) have evaluated a segmentation technique for magnetic resonance (MR) images of the brain based on the adaptive fuzzy leader clustering (AFLC) algorithm. The classifications produced using this cost function are sufficiently good since the binary image produced by the MLP can be post-processed using an active contour model to provide an accurate segmentation. Ramiro Castellanos et al. Segmentation of MR images is formulated as an unsupervised vector quantization process. closed contour. The perceptron is trained to produce a binary classification of each pixel as either a boundary or a non-boundary point.9 Network detects the suspect regions. a novel cost function based on effectiveness is proposed for training that allows us to achieve dramatic improvements in segmentation performance. the resulting binary (edge-point) image forms the external energy function for a snake. Chin-Ming Hong et al. (2006) propose a novel neuro fuzzy network which can efficiently reason fuzzy rules based on training data to solve the medical diagnosis problems. which are interpreted by the Fuzzy Neural Network of specification. used to link the candidate boundary points into a continuous.

Yue Wang et al. The image techniques used for brain segmentation could be divided in the following groups: threshold-based segmentation. In this algorithm. In the category of threshold-based segmentation. The problem is formulated as distribution learning and relaxation labeling that may be particularly useful in quantifying and segmenting abnormal brain tissues where the distribution of each tissue type heavily overlaps. the other is the relative location of neighboring pixels. use of: iterative thresholding. A histogram specification technique is applied on brain MR images before segmentation. two factors of neighborhood attraction are considered. Clarke et al. In the meanwhile.10 membership functions for the employed neuro fuzzy network. this study also presents a feature reduction scheme based on the grey-relational analysis to simplify the fuzzy rules obtained from the employed neuro fuzzy network. statistical methods for brain segmentation and region growing methods. For all these groups many methods have been proposed in the literature but no one has been widely accepted in order to be used as a general method in clinics.. Experimental results indicate that the proposed neuro fuzzy network with feature reduction can discover very simplified and easily interpretable fuzzy rules to support medical diagnosis. (2003) show the neighborhood attraction based on the traditional fuzzy c-means (FCM) clustering algorithm to improve the segmentation performance. one is the feature difference between neighboring pixels in the image. (1997) present a probabilistic neural network based technique for unsupervised quantification and segmentation of the brain tissues from magnetic resonance image. S. The method can enhance the contrast between different brain tissues. Non-brain regions of all the MR images used in this paper have already been removed. ... (1995) has surveyed and presented some segmentation methods that have been proposed in medical image community. The quantification is achieved by model-histogram fitting of probabilistic self-organizing mixtures and the segmentation by global consistency labeling through a probabilistic constraint relaxation network. The new technique utilizes suitable statistical models for both the pixel and context images. Shen et al.

the partial volume effect (when a pixel represents more than one kind of tissue type) and also some anatomical changes from one person to another. there are a lot of problems that researchers have to deal with in order to build an accurate segmentation system.2. 2. Statistical methods represent another important category in the segmentation process and most of the approaches proposed in this category use some statistical classifications combined with different image processing techniques in order to segment the MR images. Also. The total weight of an average brain is 1.2 Artificial Neural Networks 2. . more research is demanded in this domain and maybe some combination of the already existing methods or some new image processing techniques will lead to better results. so an average neuron weighs something less than 1.5kg. the use of statistical methods in texture analysis has been proposed. The segmentation problem widely encountered by researchers is the noise introduced with the acquisition of the image. Since all these techniques have their advantages and drawbacks.1 Learning from Biology A small neuron is a small cell that receives electrochemical stimuli from multiple sources and responds by generating electrical impulses that are transmitted to other neurons or effectors cells. The blood vessels could also influence image that is taken and could introduce some noise. There is something like 1010 to 1012 neurons in the human nervous system and each is capable of storing several bits of “information”. So. Some of these techniques are followed by some refinement of the segmented zones. the overlapping intensities (different brain structures have different tissue characteristics which results in various signal intensities and these intensities could overlap).5 X 10-9 g.11 histogram analysis and morphological operations are proposed.

The pulses cause a chemical neurotransmitter substance to be released at the terminating synapses which. The remaining 90% are interconnected with other neurons which store information or perform various transformations on the signals being propagated through the networks. The connections are made through two general types of synapses. They range from about 50 to a few hundred spikes per second. . The rate of pulse propagation ranges from about 5 to 125 ms-1.1 below shows a simplified biological neuron. and an axon trunk that carries an action potential output to other neurons through terminal links and synapses. The activity of a neuron is measured by the firing frequency of the potential analogue spikes which it generates. the neuron “fires” by propagating a sequence of action potential spikes down the axon to either excite or inhibit other neurons. When the cumulative inputs raise the potential above a threshold value. can excite or inhibit other neurons. This potential may be increased or decreased by the input activity received from other cells through the synapses. About 10% of neurons are input (afferent) and output (efferent). in turn. They are composed of nucleus. excitatory and inhibitory. and the time required for a stimulus to “traverse” a synapse is about 10ms during which the neuron cannot fire again. A single neuron may be connected to hundreds or even tens of thousands of other neurons. Neurons are complex cells that respond to electrochemical signals. numerous dendrite links providing input connections from other neurons through synapses.12 Neurons receive input from sensory or other types of cells and send outputs to other neurons or effectors organs such as muscles and glands. a cell body. Figure 2. Neural activity is related to the creation of an internal electric potential called a membrane potential. Although many different types of neurons have been identified. they all share some common characteristics.

2. multi sensor data fusion and noise filtering.1: Biology Neuron Artificial Neural Networks (ANNs) are imitation of simplified models of the central nervous system. data compression. ANNs have been shown to be effective as computational processors for various tasks including pattern recognition in speech and visual image recognition.2 Generic Model of Neural Network Although ANN architectures differ in several characteristic ways. associative recall.2. a sort of filter which is part of the linkage connecting the input to the . They are networks of highly interconnected neural computing elements that have the ability to respond to input stimuli and to learn to adapt to the environment. Biological neurons or networks are able to process millions of input stimuli in milliseconds even though the processes are electrochemical in nature and propagate at relatively slow milliseconds rates. Each input link i has an associated external input signal or stimulus xi and a corresponding weight wi. modeling and forecasting. combinational problem solving. a typical ANN neuron or computing element is basically a comparator that produces an output when the cumulative effect of the input stimuli exceeds a threshold value. classification. adaptive control.13 Figure 2.

1). which model the synaptic neural connections in biological nets. binary (0. The weights can also be binary or real-valued.1) 2. Normally.2. net = x1 w1 + x 2 w2 + x3 w3 = ∑ xi wi i =1 3 (2. 1). act to either increase (excitatory) or decrease (inhibitory) the input signals to the neuron.2.) producing an output y = f(net).3 Bias and Threshold A bias acts exactly like a weight and it is considered as a connection and its activation function is always 1. This is shown in Figure 2. A bias is often used to increase .2: Generic model of Artificial Neural Network The weights. The neuron behaves as an activation or mapping function f(.14 neuron. but are usually assumed to be real (positive for excitatory and negative for inhibitory links). The outputs from the ANN can also be real-valued or binary or bipolar. It is adapted similar to the way a weight is adapted according to the learning rule of the neural network. where net is the cumulative input stimuli to the neuron and f is typically a nonlinear function of net. or bipolar (-1. Figure 2. net is often taken as the weighted sum of the inputs and f is typically a monotonic non-decreasing function of net and this can be summarized in the equation below. The xi values can be real (+ or -).

then i < j and it also can be in multilayer networks. 2. Multilayered models generally use a sigmoid activation function. A threshold or normally symbolized as θ. some models of neural network do not use any bias signals. 1] or [-1. 1]. is a value that is used to make some form of decisions in an artificial neural network such that the neural network will fire or unfire. In thermodynamic models a linear threshold function is normally used. 2. However. The network is said to be a feed-forward network when the units can be labeled with integers in such a way that there is a connection from the computation unit labeled i to the computation unit labeled j. with the arcs of the digraph representing the flows of signals between units. Some of the units are termed input units: these receive signals not from other units. and therefore have continuous activations.1}.2. The choice of the activation function strongly depends on the neural network model.2.4 Activation Functions In most neural network models. and the activation values are chosen from {0.1} or {-1. such as to improve convergence. mostly chosen from [0.3 below.15 the signal levels in neural network. . Units that do not transmit signals to other units are termed output units. one of the above mentioned activation functions is used. This is illustrated in the Figure 2.5 Network A neural network is formed when we place units at the vertices of the directed graph. but instead they take their signals from the outside environment. It is quite similar to a bias but not adaptable.

generally a total error E is used that is usually chosen to be the sum of the single error values. ii.3: Feedforward Network 2. then the weights are changed in a way that a reduction of the absolute value of error.6 Learning Phase Basically. . until it has reached its inoperative mode. A pair of patterns of the learning problem is chosen and the input i presented to the neural network. iii. The learning process stops when the total error E is close enough to 0. As a stop criterion for the learning process. there are two ways of learning in artificial neural network which are supervised and unsupervised learning. The overall concept of learning is to change the weights in the neuron in such a way that the error value is minimized for each pair of patterns of the fixed learning problem. This total error is calculated anew after each epoch. An epoch is a complete run through the learning problem such that each pair of input / output patterns is processed once by the network using the learning algorithm.2. A supervised learning algorithm can be divided into five steps: i. iv.16 Figure 2. The error value (e) is determined and added to the total error E. If e ≠ 0. The input is then. The output determined by the network is compared to the target pattern. propagated through the network. or when the network is obviously not able to solve the learning problem.

unsupervised learning algorithm works on a free learning problem which only contains input patterns. In unsupervised learning algorithm. There are however. It is trained without teaching signals or targets and it is only supplied with examples of the input patterns that it will solve eventually. E is checked and when the stop condition reached. However. This theory is a result of the insufficiency of Boolean Algebra to many problems of the real world. On the other hand. then the procedure continues with step (i) and the value of E is kept.17 v.3 Fuzzy Logic Systems The theory of Fuzzy Logic was first raised by the mathematician Lotfi A. 2. If the current epoch is not yet over. total error. The most common supervised learning algorithms involve approximation of a gradient descent method and try to reduce the total error E to zero. some disadvantages of this technique as the local minimum is not the global minimum. Zadeh in 1965. for this kind of learning procedure the network is also supposed to map similar input patterns to output that are similar to each other. the weights would have been adapted in such a manner such that similar patterns are clustered into a particular node. The algorithm stops when it reaches a local minimum. because it is equivalent to a local heuristic search procedure. However. then. this learning procedure cannot always guarantee convergence. At the end of the learning phase. As most of the information in the real world is . the learning process has ended successfully. Else. This type of learning usually has an auxiliary cost function which needs to be minimized and the weights are modified where a cost function is minimized. the units compete with each other and the principle of winner-takes-all applied. When an epoch is over. it will repeat the general steps above until it converges or reaches the stop condition. This means that the minimized error is not reached and the learning process has failed.

Everything is imprecise if you look closely enough. Fuzzy systems are knowledge based or rule based systems. • Fuzzy logic is tolerant of imprecise data. indistinct. With any given system. • Fuzzy logic is flexible. imprecisely defined. A fuzzy IF-THEN rule is an IF-THEN statement in which some words are characterized by continuous membership functions. the word Fuzzy is defined as blurred. This process is made particularly easy by adaptive techniques like Adaptive Neuro-Fuzzy Inference Systems (ANFIS). The mathematical concepts behind fuzzy reasoning are very simple. most things are imprecise even on careful inspection. • Fuzzy logic can model nonlinear functions of arbitrary complexity. and one of humans' greatest abilities is to effectively process imprecise and "'fuzzy" information. which are available in the Fuzzy Logic Toolbox. According to the Oxford English Dictionary.3.1 Advantages of Fuzzy Logic Below are some reasons why we need to use fuzzy logic rather than linear system: • Fuzzy logic is conceptually easy to understand.18 imprecise. . The heart of a fuzzy system is a knowledge base consisting of the so called fuzzy IF-THEN rules. Fuzzy reasoning builds this understanding into the process rather than tacking it onto the end. What makes fuzzy nice is the "naturalness" of its approach and not its far-reaching complexity. but more than that. confused or vague. 2. it's easy to massage it or layer more functionality on top of it without starting again from scratch. • Fuzzy logic can be built on top of the experience of experts. A fuzzy system can be created to match any set of input-output data.

that which is used by ordinary people on a daily basis.2 Disadvantages of Fuzzy Logic Fuzzy logic is not a cure-all. for example. Since fuzzy logic is built atop the structures of qualitative description used in everyday language. The last statement is perhaps the most important one and deserves more discussion.3. which take training data and generate opaque. Sentences written in ordinary language represent a triumph of efficient communication. try something else. fuzzy logic lets you rely on the experience of people who already understand your system. • Fuzzy logic can be blended with conventional control techniques. Fuzzy logic is the modification of common sense . use it. In many cases fuzzy systems augment them and simplify their implementation.19 In direct contrast to neural networks. Fuzzy systems don't necessarily replace conventional control methods. We are generally unaware of this because ordinary language is. The basis for fuzzy logic is the basis for human communication. However. Natural language. . This observation underpins many of the other statements about fuzzy logic. something we use every day. 1997]. When should you not use fuzzy logic? The safest statement is the first one made in this introduction: fuzzy logic is a convenient way to map an input space to an output space. If it's not convenient. has been shaped by thousands of years of human history to be convenient and efficient. fuzzy logic is easy to use. do a fine job without using fuzzy logic. impenetrable models. Many controllers. of course. a very powerful tool can be proved for dealing quickly and efficiently with imprecision and nonlinearity. • Fuzzy logic is based on natural language. [Jang and Sun. if time is taken to become familiar with fuzzy logic.use common sense when it is implemented and the right decision will probably been made. If a simpler solution already exists. 2.

n) of A are together a subset of the universal base set X.4) Each fuzzy set is completely and uniquely defined by one particular membership function..2) If these elements ai (i = 1. Therefore. . so two values of truth.. The notation for the membership function.4 below. a n } (2.3 Fuzzy Sets In the classical set theory a set can be represented by enumerating all its elements using A = {a1 . a3 . A : X → [0. The membership of a fuzzy set is described by this membership function µA(x) of A which associates to each element xo = X a grade of membership µA(xo). such as big. µA(x) of a fuzzy set A is as equation 2. a 2 ....4 shows the differences in the crisp and fuzzy sets.20 2. This function is a generalization of that in equation 2..1].3. into real numbers in [0.. small and others. for which also a characteristic function can be defined.3) In classical set theory µA(x) has only the values 0 (``false'') and 1 (``true''). Such sets are also called crisp sets..3 and called a membership function... Figure 2. the set A can be represented for all elements x∈ X by its characteristic function µ A (x) =  1 0 if x ∈ A otherwise (2. In contrast to classical set theory a membership function µA(x) of a fuzzy set can have in the normalized closed interval [0 1].. each membership function maps elements of a given universal base set X which is itself a crisp set. Non-crisp sets are called fuzzy sets.1] (2... Consequently symbols of membership functions are also used as labels of the associated fuzzy sets.

rule base and defuzzification. Figure 2.5: Membership grades of a fuzzy set 2. such as triangular or trapezoidal functions.3. . This value is in the unit interval [0.1].5.6 shows the components of a fuzzy system.4: Membership functions of a crisp set and a fuzzy set 2. In some cases. inference engine.5 Components of a fuzzy system There are basically four components in the fuzzy logic system which is fuzzification.21 x Figure 2.3. whereby for µA(x) a large class of functions can be taken. Reasonable functions often used are linear functions. a fuzzy system is drawn as a black box with some inputs and an output. The grade of membership µA(x) of a membership function describes which grade it belongs to in the fuzzy set A. x Figure 2.4 Membership functions The membership function µA(x) describes the membership of the elements x of the base set X in the fuzzy set A. This is shown in Figure 2.

6: Components of a Fuzzy Logic System The first block inside the controller is fuzzification. In composition based inference. Fuzzification is used to convert each piece of input data to degrees of membership by a lookup in one or several membership functions. all rules in the fuzzy rule base is combined into a single fuzzy relation which is then viewed as a single fuzzy IF-THEN rule.22 Figure 2. Inference engine simply implies the combination of certain rules into a mapping from a fuzzy rule base to get some output corresponding to the inferred rules. Most common type of inference used is shown below:- • The fuzzy intersection operator ∩ (fuzzy AND connective) applied to two fuzzy sets A and B with the membership functions µA(x) and µB(x) is µ A∩ B ( x) = min{µ A ( x). There is a degree of membership for each linguistic term that applies to that input variable. It is the heart of the fuzzy system in the sense that all other components are used to implement these rules in a reasonable and efficient manner. µ B ( x)}. x∈ X (2.5) . A fuzzy rule base consists of a set of fuzzy IF-THEN rules. The fuzzification block thus matches the input data with the conditions of the rules to determine how well the condition of each rule matches that particular input instance. There are two ways to infer with a set of rules which are composition based inference and individual rule based inference.

Simple computation of fuzzy inference system is also one of the issues that need to be considered besides the intuitive issue and some inference need special properties. x∈ X (2. For intuitive appeal. centre of singleton (COS).23 • The fuzzy union operator ∪ (fuzzy OR connective) applied to two fuzzy sets A and B with the membership functions µA(x) and µB(x) is µ A∪ B ( x) = max{µ A ( x). The value inferred by the fuzzy rule base is in the fuzzy value or in other words. each rule in the fuzzy rule base determines an output fuzzy set and the output of the whole fuzzy inference engine is the combination of many individual fuzzy sets. the choice should make sense from an intuitive point of view. x∈ X (2.7) In individual rule base inference. It is not a unique operation as different methods are possible. computational efficiency and special properties. .6) • The fuzzy complement operator (fuzzy NOT operation) applied to the fuzzy sets A with the membership functions µA(x) is µ A ( x) = 1 − µ A ( x). The most common are centre of gravity (COG). µ B ( x)}. weight average and mean-max membership method. In general. Defuzzification is used to transform fuzzy information into a single value. for every possible value µ. The combination can be taken either by union or by intersection. one gets a grade of membership that describes to what extent this value µ is reasonable to use. maximum membership. there are three criteria of choosing the inference methods which are intuitive appeal.

6 Fuzzy Logic Graphical User Interface (GUI) There are five primary GUI tools for building.24 2.3. 4.7: Fuzzy Logic Graphical User Interface (GUI) . 3. 5. and observing fuzzy inference systems in the Fuzzy Logic package: 1. editing. 2. Fuzzy Inference System (FIS) Editor Membership Function Editor Rule Editor Rule Viewer Surface Viewer Figure 2.

and the last of these variables can be removed with minimum loss of real data. 2. The first principal component is the combination of variables that explains the greatest amount of variation. The uncorrelated variables are linear combinations of the original variables. the FIS. Essentially. The Rule Viewer is a MATLAB technical computing environment based display of the fuzzy inference diagram shown at the end of the last section. Fuzzy Logic Toolbox software does not limit the number of inputs. The Rule Viewer and the Surface Viewer are used for looking at. it generates and plots an output surface map for the system. The Membership Function Editor is used to define the shapes of all the membership functions associated with each variable.25 The FIS Editor handles the high-level issues for the system. The Surface Viewer is used to display the dependency of one of the outputs on any one or two of the inputs-that is. It computes a compact and optimal description of the data set. as opposed to editing. a set of correlated variables are transformed into a set of uncorrelated variables which are ordered by reducing variability. or how individual membership function shapes are influencing the results.4 Principal Component Analysis (PCA) Principal Components Analysis (PCA) is a multivariate procedure which rotates the data such that maximum variability is projected onto the axes. The main use of PCA is to reduce the dimensionality of a data set while retaining as much information as is possible. If the number of inputs is too large. the number of inputs may be limited by the available memory of the machine. then it may also be difficult to analyze the FIS using the other GUI tools. However. The Rule Editor is for editing the list of rules that defines the behavior of the system. Used as a diagnostic. or the number of membership functions is too big. The second principal component defines the next . They are strictly read-only tools. it can show (for example) which rules are active.

The third axis contains the maximum amount of variation orthogonal to the first and second axis and so on until one has the last new axis which is the last amount of variation left. . There can be as many possible principal components as there are variables. or "accounts" for the maximum amount of variation. etc. The second axis contains the maximum amount of variation orthogonal to the first. The first new variable contains the maximum amount of variation. The criteria being that the first axis "contains" the maximum amount of variation. the second new variable contains the maximum amount of variation unexplained by the first and orthogonal to the first. there will be no correlation between the new variables defined by the rotation. It can be viewed as a rotation of the existing axes to new positions in the space defined by the original variables. In this new rotation.26 largest amount of variation and is independent to the first principal component. It can be viewed as finding a projection of the observations onto orthogonal axes contained in the space defined by the original variables.

CHAPTER 3

RESEARCH METHODOLOGY

3.1

Methodology

Basically, two main phases are employed in this project which is theoretical phase and practical phase. Theoretical phase involves the process of reading and understanding, reviewing of theories, studying and surveying the ANFIS model, classification and detection techniques on medical image fields.

On the other hand, the practical phase involves medical image processing, neuro fuzzy model coding, simulation, assumptions, adaptation and training of the neuron and etc by using MATLAB.

3.2

Proposed Method

There are 7 stages involved in the proposed model which starts from the data input to output. The first stage is the image processing system. Basically in image processing system, image acquisition and enhancement are steps that need to be done. In this project, these two steps are skipped and all the images are collected from available resources. The proposed model requires converting the image into a format capable of being manipulated by the computer. The MR images are converted into matrices form by using MATLAB. Then, the neuro fuzzy model is developed using MATLAB programming. After the neuro fuzzy model is successfully developed, the classification of the MR images starts.

28 Symptoms detection phase will follow once the output from the classification technique is done and lastly, performance based on the result will be analyzed at the end of the development phase. The proposed brain MR images classification and detection method is shown in Figure 3.1.

MR Image Input

MATLAB

Data Form

Neuro Fuzzy Model

Classification & Detection

Output

Performance Analysis Figure 3.1: The proposed system

3.3

Transformation of Image to Data Form

The MR image will obtain and convert it to data form (encoded information that can be stored, manipulated and transmitted by digital devices) in MATLAB environment such as basic arithmetic operations (addition, subtraction, and multiplication) and indexing, including logical indexing reshaping, reordering, and concatenating. The basic data structure in MATLAB is the array, an ordered set of real or complex elements. This object is naturally suited to the representation of images, real-valued ordered sets of color or intensity data. MATLAB stores most images as two-dimensional arrays (i.e., matrices), in which each element of the

29 matrix corresponds to a single pixel in the displayed image. (Pixel is derived from picture element and usually denotes a single dot on a computer display.) For example, an image composed of 200 rows and 300 columns of different colored dots would be stored in MATLAB as a 200-by-300 matrix. This convention makes working with images in MATLAB similar to working with any other type of matrix data, and makes the full power of MATLAB available for image processing applications. An intensity image is a data matrix, whose values represent intensities within some range. MATLAB stores an intensity image as a single matrix, with each element of the matrix corresponding to one image pixel. The matrix can be of class double, uint8, or uintl6.

A simple program (shown in Appendix) is created to transform images. dot of pixel labelling and also region of interest extraction. Initially, this program asked the user to insert normal brain image as the prior image and then insert an abnormal brain image as a diagnostic image. Both images are converted to matrix format automatically while the image being read in the MATLAB environment. Both images are obtained in the RGB format. An RGB image is stored in MATLAB as an m-by-n-by-3 data array that defines red, green, and blue color components for each individual pixel. If it is difficult to manipulate, it must be converted to intensity format. An intensity image is a data matrix, whose values represent intensities within some range. MATLAB stores an intensity image as a single matrix, with each element of the matrix corresponding to one image pixel. The elements in the intensity matrix represent various intensities, or gray levels, where the intensity 0 usually represents black and the intensity 1, 255 or 65535 usually represent full intensity, or white. Both intensity of normal brain and abnormal brain images are converted to binary images as shown in Figure 3.2.

The 'bwarea' function is used to convert the area to binary image. The area is a measure of the size of the foreground of the image. In other words, the area is the number of on pixels in the image. Comparison between the areas of the normal and the abnormal brain in a binary image is a technique used to make sure the size of both brain images are the same.

as illustrated in Figure 3.3. the first component r (the row) increases downwards. ordered from top to bottom and left to right.4 Pixels Labeling In succession. Pixel coordinates are integer values and range between 1 and the length of the row or column. This correspondence makes the relationship . while the second component c (the column) increases to the right. There is a one-to-one correspondence between pixel coordinates and the coordinates MATLAB uses for matrix subscripting.3: Pixel coordinate system For pixel coordinates.30 Figure 3. every dot of pixel will be labeled in different coordinates. the most convenient method for expressing locations in an image is to use pixel coordinates. Figure 3.2: Convert brain image to binary image 3. Under this coordinate system. Generally in the Matlab environment. the image is treated as a grid of discrete elements.

4 shows the advanced coordinate system for matrix 16 by 16.5 Region of Interest Extraction The region of interest is determined by subtracting the normal brain image with abnormal image brain image as shown in Figure 3. first column is stored in the single number 1. advanced coordinate system will be created. For instance. however pixel in the first row. For example. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 Figure 3. second column is stored in the single number 2 and so on. 2). second column is stored in the matrix element (5.5. By using the existing pixel coordinate system.4: Advanced coordinate system for matrix 16 by 16 3. .31 between an image's data matrix and the way the image is displayed easy to understand. every dot of pixel coordinate in a single number. the data for the pixel in the fifth row. It means that. pixel in the first row. Figure 3.

. The purpose of PCA is to reduce the large dimensionality of the data. or 65535 usually represents full intensity.32 Normal Brain Abnormal Brain Region of Interest Figure 3. The principal component analysis (PCA) is one of the most successful techniques that have been used in image recognition and compression. 3.5: Location of tumor The elements in the intensity matrix represent various intensities.6 Feature Extraction using Principal Component Analysis (PCA) In this project. 255. or white. the principal component analysis (PCA) is used as a feature extraction algorithm. where the intensity 0 usually represents black and the intensity 1. or grey levels.

In this project. j is computed the similarities between ωj and all of the feature vectors ωi’s in the training set are computed. For each training image i. Thus. In the training phase. otherwise if i ≠ j. In this project. Let 1 be a training image of image 1 which has a pixel resolution of M 1. the feature vector ωj of the test image using PCA. . In order to extract PCA features of first convert the image into a pixel vector Φ1 by concatenating each of the M rows into a single vector.33 MR image recognition systems find the identity of a given test image according to their memory. The identity of the most similar ωi is the output of the image recognizer. In the testing phase. The memory of an MR image recognizer is generally simulated by a training set. it means that the MR image j has misclassified. The similarity between feature vectors is computed using Euclidean distance. If i = j.6. these feature vectors ωi are calculated and stored. N columns). the PCA algorithm is used as a dimensionality reduction technique which transforms the vector Φ1 to a vector ω1 which has a dimensionality d where d << M x N. the task of the MR image recognizer is to find the most similar feature vector among the training set to the feature vector of a given test image. it means that the MR image j has correctly identified. feature vectors are extracted for each image in the training set. x N (M rows. Schematic diagram of the MR image recognition system that is implemented is shown in Figure 3. In order to identify the test image j. The length (or. dimensionality) of the vector Φ1 will be M x N. the training database consists of a set of MR images.

This project uses the ANFIS Editor GUI menu bar to load a FIS training initialization.6: Schematic diagram of a MR image recognizer. a training data set that contains the desired input/output data pairs of the targeted system is modeled. From this GUI: l. 3. Load data (training) by selecting the appropriate radio buttons in the Load data portion of the GUI and then clicking Load Data.7 Data Modeling Eventually. The loaded data is plotted on the plot region. They fully represent the features of the data that the trained FIS intends to model.34 Figure 3. . and then save the trained FIS. The coordinated abnormal tissues which are in the matrix form are presented to ANFIS for training (estimating) membership function parameters.

5. the number of rows of the training data is equal to the number of training data pairs. and. Therefore.7 shows the training data set for this research.35 2. This function plots the rest data against the FIS output in the plot region. 3. Each row of the training data is a desired input/output pair of the target system to be modeled. . This research uses the default hybrid method as the FIS model parameter optimization method. and testing data output by clicking the Test Now button. View the FIS model structure once an initial FIS has been generated or loaded by clicking the Structure button. the number of columns training data is equal to the number of inputs plus one. Choose the FIS model parameter optimization method backpropagation or a mixture of backpropagation and least squares (hybrid method). Each row starts with an input vector and is followed by an output value. 6. Generate an initial FIS model or load an initial FIS model using the options in the Generate FIS portion of the GUI. Figure 3. 4. Choose the number of training epochs and the training error tolerance. This research uses 0 as training epochs and 40 as training error tolerance. View the FIS model output versus the training. since there is only one output. 7. The training data is a required argument to the ANFIS Editor GUI. This training adjusts the membership function parameters and plots the training error plot(s) in the plot region. Train the FIS model by clicking the Train Now button. checking.

7: Training Data Set After data modeling by ANFIS.36 Input vector Output Figure 3. use the function evalfis. where M represents the number of input values specified above. Output: the output matrix of size M-by-L.fis file on the disk and bring the resulting file into workspace. then evalfis takes each row of input as an input vector and returns the M-by-L matrix to the variable. To evaluate the output of a fuzzy system for a given input. read this fuzzy inference system from an anfis.8 shows the Basic flow diagram of computations or algorithm in ANFIS: . The ‘evalfis’ has the following arguments. where each row is an output vector and L is the number of output variables. output. Figure 3. • Input: a number or a matrix specifying input values. If input is an M-by-N matrix. • • Fismat: a FIS structure to be evaluated. and L is the number of output variables for the FIS. where N is a number of input variables.

all files are placed in a folder included in the MATLAB working path.8: Basic flow diagram of computations in ANFIS 3. To work with m-files and data.1 Data Management and Preparation As this project fully use the MATLAB software.8. The most practical solution for beginners is to place files in the folder ‘…/MATLAB/Work’. The working path is accessible by clicking the ‘Path Browser’ button in the MATLAB command window. it is advisable to know its data management system.8 System developing 3.37 Initialize the fuzzy system Use genfis1 command Give other parameters for learning Important are: Number of iterations (epochs) Tolerance (error) Start learning process Use anfis command Stop when tolerance is achieved Validation data Use evalfis command Figure 3. or typing ‘path’ in the prompt. usually the . Any file located outside this path will be invisible to MATLAB. The working path is a collection of folders where MATLAB searches files when these are called.

checking data . classification and FIS phase. But we recommend beginners to work with the ‘…/MATLAB/Work’ folder as the working directory.e. Training data means that the data is used for ANFIS training and checking on the other hand.2 Data Mining The data are provided from the transformation MR images into data form as explained before. The data is divided into 3 partition which are training. Application data is a collection of untrained data needed in testing the accuracy of the model. 3. the user is required to enter the files that have been partitioned accordingly by keying the file name in the space provided. any data file or m-file should be named properly. is used to check the model created. An error will occur if m-files files are not in its directory. and only edit the MATLAB working path if either this folder is absent or it is not the current directory.8. They are training data file. In this package development. the folder where MATLAB preferentially loads or saves data).3 Input Phase In this phase. ANFIS phase. 3. checking takes around 5 patterns and the rest which is 25 patterns are used for application purposes.38 default current directory (i. output phase (application and detection) and is explained in details below. The main program is developed in 4 phases which are input phase. MATLAB is case sensitive.8. Thus. It means that the current directory is not in the folder. checking and application. 4 main input files are needed. Training data has 20 patterns. It has a total of 50 data with 3 inputs and 1 output. The MATLAB working path is easily edited with the ‘Path Browser’ tool.

9: Partial of input phase 3. User also is required to enter the total number of cluster and epochs required for training purposes. it is advisable to contain the data only and not combined together with training target. Genfis1 basically generates initial fuzzy inference system for ANFIS training by first applying classification technique on the data.8. It is always an easy way to paste data from a spreadsheet in the Windows Notepad and saving it under extension ‘dat’. Figure 3. All data and m-files should be placed in a folder included in the MATLAB working path. On the other hand. the checking file that is used for model checking and validation should contain data and training target together. relevant data files will be loaded for Fuzzy Inference System (FIS) building.4 FIS Phase After the input phase. Figure 3. The fuzzy inference system is summarized in a function called genfis1.9 shows a partial of the input phase and full coding can be accessed in Appendix. Application data purely contain the untrained data and is subjected to test the accuracy of the model. Genfis1 function . arranged in rows and columns. A convenient format to work with MATLAB is ASCII. target data file and application data file including expected result data file. For the training data file.39 file.

Information regarding the ANFIS model will prompt at the early of the process as shown in Figure 3. The second column shows the training error and third column indicates the checking error. Figure 3.40 requires 3 main inputs which are training data without target. . Output from genfis1 is considered as initial information for building the fuzzy inference system which is useful for ANFIS training. Figure 3. The data such as number epoch and fuzzy inference system (FIS) is used to train and validate the ANFIS model.11.10 shows a sample of generated fuzzy inference. This is accomplishes by extracting a set of rules that models the data behavior.8. target data and number of membership function required.10: FIS generated thru genfis1 3. Figure 3.12 shows the errors regarding to the number of epoch and the total time required for ANFIS training. Result from training error and checking error will be calculated depending on the step size and is in matrix form.5 ANFIS modeling phase This phase is fully the training of ANFIS model.

12: ANFIS training errors and total time training .11: Information of the ANFIS model Figure 3.41 Figure 3.

By making comparison between the expected result and output from the ANFIS model. Figure 3.13 shows an example of the result.6 Output phase The output phase involves the application of the model to observe how well the model reacts to the untrained data and also the detection of abnormal tissue based on the data itself. Figure 3.8. User is required to enter the application file name and also the expected result. the class of the abnormal tissue can be determined.42 3.13: Sample of classification results .

CHAPTER 4 NEURO FUZZY MODELING AND SOFTWARE IMPLEMENTATION 4. . neural network learning algorithms are usually gradient descent methods. An even more important aspect is that the system should always be interpretable in terms of fuzzy if-then rules.1 Neuro Fuzzy Model A neuro fuzzy system is a combination of neural network and fuzzy systems in such a way that neural network or neural network algorithms are used to determine the parameters of the fuzzy system. However. This means that the main intention of neuro fuzzy approach is to create or improve a fuzzy system automatically by means of neural network methods. In order to realize the system. because the functions used to realize the inference process are usually not differentiable. Then a learning algorithm such as back propagation is used to train the system. The idea of a neuro fuzzy system is to find the parameters of a fuzzy system by the means of learning methods obtained from neural network. we need to replace the functions used in the fuzzy system (like min and max) by differentiable functions or do not use a gradient-based neural learning algorithm but a better-suited procedure. A common way to apply a learning algorithm to a fuzzy system is to represent it in a special neuralnetwork-like architecture. because it is based on a fuzzy system reflecting vague knowledge. This cannot be applied directly to a fuzzy system.

4. The fuzzy rules encoded within the system represent vague samples. and it has nothing to do with fuzzy logic in the narrow sense. a neuro fuzzy system should not be seen as a kind of (fuzzy) expert system. for example the membership functions bounds. and causes only local modifications in the underlying fuzzy system. It is possible both to create the system out of training data from scratch. a neural-fuzzy system can always be interpreted as a system of fuzzy rules. . The learning procedure of a neural-fuzzy system takes the semantical properties of the underlying fuzzy system into account. The units in this network use t-norms or t-conorms instead of the activation functions normally used in neural networks. It can be viewed as a special kind of feed forward neural network.2 Adaptive neuro-fuzzy inference system (ANFIS) A neuro-fuzzy (ANFIS) system is a combination of neural network and fuzzy systems in such a way that neural network is used to determine the parameters of fuzzy system. This results in constraints on the possible modification of the system’s parameters. Fuzzy sets are encoded as (fuzzy) connection weights. It also approximates an ndimensional (unknown) function that is partially given by the training data. Generally. leading to improved performance without operator invention. but datadriven. and to initialize it from prior knowledge in the form of fuzzy-rules. ANFIS largely removes the requirement for manual optimization of the fuzzy system parameters. and can be viewed as vague prototypes of the training data. A neural network is used to automatically tune the system parameters. A neuro fuzzy system is a fuzzy system that is trained by a learning algorithm (usually) derived from neural network theory. The (heuristic) learning procedure operates on local information.44 Modern neuro fuzzy systems are often represented as multilayer feed forward neural network. Besides. The learning process is not knowledge-based.

2. using a neuro fuzzy scheme.1 ANFIS Architecture The ANFIS is a fuzzy Sugeno model put in the framework of adaptive systems to facilitate learning and adaptation (Jang. 1993). Figure 4. 4. In neural network the training essentially builds the system.1 illustrates the reasoning mechanism for this Sugeno model where it is the basis of the ANFIS model.45 The neuro fuzzy system with the learning capability of neural network and with the advantages of the rule-base fuzzy system can improve the performance significantly and can provide a mechanism to incorporate past observations into the classification process. However. pi. Ai and Bi are the fuzzy sets. Very fast convergence time. Some advantages of ANFIS are: • • Refines fuzzy if-then rules to describe the behaviour of a complex system. To present the ANFIS architecture. the system is built by fuzzy logic definitions and is then refined using neural network training algorithms. two fuzzy if-then rules based on a first order Sugeno model are considered: Rule 1: If (x is A1) and (y is B1) then (f1 = p1x + q1y + r1) Rule 2: If (x is A2) and (y is B2) then (f2 = p2x + q2y + r2) where x and y are the inputs. Does not require prior human expertise o uses membership functions plus o desired dataset to approximate Greater choice of membership functions to use. qi and ri are the design parameters that are determined during the training process. fi are the outputs within the fuzzy region specified by the fuzzy rule. Such framework makes the ANFIS modeling more systematic and less reliant on expert knowledge. . 1992.

2. Adaptive neuro fuzzy inference system basically has a 5 layer architectures and each of the function is explained in detail later. 4.46 Figure 4. Figure 4. which are given by: O1.1) (4. The outputs of layer 1 are the fuzzy membership grade of the inputs. In the first layer.i = µ Ai (x) O1.2) . whereas a square indicates an adaptive node.2: ANFIS architecture.i = µ Bi−2 (y) i = 1. (4. all the nodes are adaptive nodes.1: A two-input first-order Sugeno fuzzy model with two rules The ANFIS architecture to implement these two rules is shown in Figure 4. in which a circle indicates a fixed node. 2 i = 3.

In the fourth layer. µ Bi−2(y) can adopt any fuzzy membership function. if the bell shaped membership function is employed. µ Ai (x) is given by: µ A ( x) = 1 x − ci 1+ ai 2b (4.i = wi f i = wi ( pi x + q i y + ri ) i = 1.i = wi = µ Ai (x)µ Bi (y) i = 1. The outputs of this layer can be represented as: O2. In the third layer. the nodes are fixed nodes. the nodes are adaptive. The output of this layer can be represented as: O3. The output of each node in this layer is simply the product of the normalized firing strength and a first order polynomial (for a first order Sugeno model). to indicate that they play a normalization role to the firing strengths from the previous layer. 2. For example. In the second layer.2. Thus. They are labeled with π.5) which are the so-called normalized firing strengths.i = wi = wi . (4. the nodes are also fixed nodes labeled by N.6) . governing the bell shaped functions accordingly. wi + w2 i = 1. bi and ci are the parameters of the membership function.4) which are the so-called firing strengths of the rules.3) where ai. the output of this layer is given by: O4.47 where µ Ai(x). indicating that they perform as a simple multiplier. 2 (4. (4.

where each input is assumed to have three associated MFs. This node performs the summation of all incoming signals. pertaining to the first order polynomial.4 illustrates how the two dimensional input space is partitioned into nine overlapping fuzzy regions. 1992. while the consequent part specifies the output within the region. In the first layer. the premise part of a rule defines a fuzzy region. In the fourth layer.7) It can be observed that there are two adaptive layers in this ANFIS architecture. namely the first and the fourth layers. 1993).i = ∑ wi f i = i ∑wf ∑w i i i i i (4. there is only one single fixed node labeled with ∑. In other words. there are three modifiable parameters {ai. there are also three modifiable parameters {pi. These parameters are the so-called premise parameters. bi. These parameters are the so-called consequent parameters (Jang. Figure 4. Figure 4. qi. . ci}. ri}. Hence. each of which is governed by a fuzzy if-then rule.48 In the fifth layer. the overall output of the model is given by: O 5.3 shows the variation in the Sugeno model that is equivalent to a two-input first-order Sugeno fuzzy model with nine rules. which are related to the input membership functions.

namely {ai. the output of the ANFIS model can be written as: . qi.49 Figure 4. to make the ANFIS output match the training data.4: The input space that are partitioned into nine fuzzy regions 4. When the premise parameters ai. ri}. bi. ci} and {pi.2 Learning algorithm of ANFIS The task of the learning algorithm for this architecture is to tune all the modifiable parameters.2.3: Two-input first-order Sugeno fuzzy model with nine rules Figure 4. bi and ci of the membership function are fixed.

it becomes: f = w1 ( p1 x + q1 y + r1 ) + w2 ( p 2 x + q 2 y + r2 ) (4. r1. q1.8) Substituting Eq. It has been proven that this hybrid algorithm is highly efficient in training the ANFIS (Jang. p2. A hybrid algorithm combining the least squares method and the gradient descent method is adopted to solve this problem.11) which is a linear combination of the modifiable consequent parameters p1. 1993).10) After rearrangement. When the premise parameters are not fixed. q2 and r2. . The gradient descent method (backward pass) is used to adjust optimally the premise parameters corresponding to the fuzzy sets in the input domain.50 f = w1 w2 f1 + f2 w1 + w2 w1 + w2 (4. Table 4. The hybrid algorithm is composed of a forward and a backward pass.8) yields: f = w1 f 1 + w2 f 2 (4. The output of the ANFIS is calculated by employing the consequent parameters found in the forward pass. the search space becomes larger and the convergence of the training becomes slower. (5) into Eq.9). the output can be expressed as: f = ( w1 x) p1 + ( w1 y )q1 + ( w1 )r1 + ( w2 x) p 2 + ( w2 y )q 2 + ( w2 )r2 (4. The least squares method (forward pass) is used to optimize the consequent parameters with the premise parameters fixed.9) Substituting the fuzzy if-then rules into Eq. Once the optimal consequent parameters are found. 1992. the backward pass starts immediately. The output error is used to adapt the premise parameters by means of a standard backpropagation algorithm. The least squares method can be used to identify the optimal values of these parameters easily. (4.1 summarizes the activities in each pass. (4.

To achieve good generalization of unseen data. Gradient descent and Backpropagation algorithms are used to adjust the parameters of membership functions (fuzzy sets) and the weights of defuzzification (neural networks) for fuzzy neural networks. Functionally there are almost no constrains on the node functions of an adaptive network except for the requirement of piecewise differentiability. the size of the training data set should be at least as big as the number of modifiable parameters in ANFIS. provided that sufficient hidden neurons and fuzzy rules are available. . ANFIS employs gradient descent algorithm to fine-tune them. ANFIS applies two techniques in updating parameters. This approach is thus called hybrid learning method since it combines gradient descent algorithm and least-squares method.2. For consequent parameters that define the coefficients of each equation. ANFIS uses the leastsquares method to identify them. The neurons in ANFIS have different structures. The advantage of fuzzy is that prior knowledge is represented into a set of constraints to reduce the optimization research space.1: Two Passes in the Hybrid Learning Procedure for ANFIS 4. For premise parameters that define membership functions.3 ANFIS Classifier Both neural network and fuzzy logic are universal estimators. They can approximate any function to any prescribed accuracy. The adaptation of back propagation to structured network so as to automate fuzzy control parametric tuning is utilized from NN. The ANFIS is a FIS implemented in the framework of an adaptive fuzzy neural network.51 Forward pass Premise parameters Consequent parameters Signals Fixed Least-squares estimator Node outputs Backward pass Gradient descent Fixed Error signals Table 4. It combines the explicit knowledge representation of a FIS with the learning power of ANNs. The objective of ANFIS is to integrate the best features of fuzzy systems and neural network.

4. • The Normalization is by Sum and arithmetic division. save the trained FIS. . that is. • The rules are differentiable T-norm usually product. and Output (Algebraic Sum). Any data set is loaded into the ANFIS Editor GUI. • Functions are linear regressions and multiplication with w .3 ANFIS editor GUI The ANFIS Editor GUI menu bar can be used to load a FIS training initialization.5.52 • The Membership function is defined by parameterized soft trapezoids (Generalized Bell Functions). (or that is applied to the command-line function ANFIS) must be a matrix with the input data arranged as vectors in all but the last column. A sample of ANFIS Editor GUI with input is shown in Figure 4. normalized weights ω. The output data must be in the last column. open a new Sugeno system or any of the other GUIs to interpret the trained FIS model.

53 Figure 4. and those of each output variable on the right. .6. There is a simple diagram at the top that shows the names of each input variable on the left. This is shown in Figure 4.4 FIS Editor The FIS Editor displays general information about a fuzzy inference system. The sample membership functions shown in the boxes are just icons and do not depict the actual shapes of the membership functions.5: ANFIS Editor GUI 4.

7 shows the membership function editor. .5 Membership Functions Editor The Membership Function Editor shares some features with the FIS Editor. all of the five basic GUI tools have similar menu options. In fact. Figure 4. and Help and Close buttons.54 Figure 4.6: FIS Editor 4. The Membership Function Editor is the tool that lets you display and edits all of the membership functions associated with all of the input and output variables for the entire fuzzy inference system. status lines.

6 Rule Editor The Rule Editor allows user to construct the rule statements automatically.8. by clicking the appropriate button. or added. . Choosing none as one of the variable qualities will exclude that variable from a given rule. deleted. Choosing not under any variable name will negate the associated quality. Rules may be changed.55 Figure 4. by clicking on and selecting one item in each input variable box.7: Membership Function Editor 4. one item in each output box. and one connection item as in Figure 4.

Since it plots every part of every rule. it presents a sort of micro view of the fuzzy inference system. for a relatively small number of inputs and outputs. it performs well (depending on how much screen space we devote to it) with up to 30 rules and as many as 6 or 7 variables. The Rule Viewer also shows how the shape of certain membership functions influences the overall result.7 Rule Viewer The Rule Viewer allows users to interpret the entire fuzzy inference process at once. .8: Rule Editor 4. but. The Rule Viewer shows one calculation at a time and in great detail. In this sense.56 Figure 4. it can become unwieldy for particularly large systems.

A GUI can be seen in Figure 4.57 Figure 4. The node represents a normalization factor for the rules. the model structure can be viewed by clicking the Structure button in the middle of the right side of the GUI. The branches in this graph are color coded.8 ANFIS Model Structure After the FIS is generated. Color coding of branches characterize the rules and indicate whether or not and. . or or are used in the rules.9: Rule Viewer 4. The input is represented by the left-most node and the output by the right-most node. Clicking on the nodes indicates information about the structure.10. not.

11 and 4. and so on) is hypothesized.58 Figure 4.10: ANFIS Model Structure 4. In general.9 Modeling Data through ANFIS The modeling approach used by ANFIS is similar to many system identification techniques. ANFIS can then be used to train the FIS model to emulate the training data presented to it by modifying the membership function parameters according to a chosen error criterion. This is . this type of modeling works well if the training data presented to ANFIS for training (estimating) membership function parameters is fully representative of the features of the data that the trained FIS intend to model. Figures 4. a parameterized model structure (relating inputs to membership functions to rules to outputs to membership functions. First. Next.12 show the example training error in ANFIS and output of the ANFIS. input/output data is collected in a form that will be usable by ANFIS for training.

59 not always the case.11: Training Error . Figure 4. data is collected using noisy measurements. [Jang. however. In some cases. This is where model validation comes into play. 1993]. and the training data cannot be representative of all the features of the data that will be presented to the model.

60 Figure 4. . are presented to the trained FIS model. This other type of validation data set is referred to as the checking data set and it set is used to control the potential for the model over fitting the data. and its use is described in a subsection that follows. to see how well the FIS model predicts the corresponding data set output values.10 Model Validation (Checking and Testing Data) Model validation is the process by which the input vectors from input/output data sets on which the FIS was not trained. the FIS model having parameters associated with the minimum checking data model error is then selected. This is accomplished with the ANFIS Editor GUI using the so-called testing data set. When the checking and training data are presented to ANFIS. Another type of data set can also be used for model validation in ANFIS.12: Output of ANFIS 4.

61 One problem with model validation for models constructed using adaptive techniques is selecting a data set that is both representative of the data the trained model intends to emulate. If a large amount of data has been collected. yet sufficiently distinct from the training data set so as not to render the validation process trivial. . hopefully this data contains all the necessary representative features. so the process of selecting a data set for checking or testing purposes is made easier.

1: Training and Checking Data . The training data set was used to train the ANFIS. The data set was divided into two separate data sets – the training data set (20 subjects) and the testing data set (15 subjects).1 which is in the form of bar chart.CHAPTER 5 RESULTS AND DISCUSSIONS 5. Figure 5. The partition of data used is shown in Figure 5.1 Result Various experiments were performed and the sizes of the training and testing sets were determined by taking into consideration the classification accuracies. whereas the testing data set was used to verify the accuracy and the effectiveness of the trained ANFIS model for the detection of brain tumors.

01. The ANFIS was implemented by using MATLAB software package.1: The ANFIS Errors . The ANFIS used 20 training data in 40 training periods and the step size for parameter adaptation had an initial value of 0.1296 Global Error 0.2.1. triangular and pi) shaped membership function.63 The classifier is able to differentiate between normal and abnormal brain. At the end of 40 training periods. The overall result regarding to the ANFIS errors can be seen in Table 5.2: ANFIS Errors Type of error Average Training Error 0. Figure 5. There were a total of 27 fuzzy rules in the architecture of the ANFIS using a 3-type (generalized bell.0667 Table 5. the final error convergence value is 2.5x10-3 as shown in Figure 5.0038 Checking Error 0.

namely.3. all of the features used in the descriptions of instances may have different levels of relevancy. small. Figure 5. Figure 5. medium.64 5.2 Performance Analysis In a real world domain. The outputs of ANFIS are shown in Figure 5.4 shows the initial and final membership function of inputs 1. The proposed method gives a framework to detect more brain tumors using ANFIS method for the classification. Membership function of each input parameter was divided into three regions. The examination of initial and final membership functions indicates that there are considerable changes in the final membership functions but the change is very small due to less the number of data training. 2 and 3 using the generalized bell. triangular and pi shaped membership function. and large. Therefore. just like the one used in the present study. in the present study changes of the final (after training) membership functions with respect to the initial (before training) membership functions of the input parameters were examined.3: The misclassified output of ANFIS classifier .

4: a) Membership function before training b)Membership function after training. .65 a) b) Figure 5.

3 Noise Consideration The training data set in this project is 0% noise implemented. 5.3.33% and 96%.33% 96% Accuracy 94.66 After training. sensitivity and accuracy. As seen from that table. The outputs of ANFIS are shown in Figure 5. respectively. 1 subject was classified incorrectly by the ANFIS. According to Table 5.2. .5 to 5.67% Table 5. 15 testing data was used to validate the accuracy of the ANFIS classifier for the detection of brain tumors.2: The ANFIS Test Results Statistical parameters Specificity Sensitivity Values 93.7. The test results of the ANFIS are presented in Table 5. 2% and 3% are implemented to the testing data set to look the performance of the ANFIS classifier with noise consideration. Results from the ANFIS classifier are all displayed as a summary of the performance on the test data. Type of Data MRI Iris Number of subjects 15 Number of subjects classified correctly 14 25 24 Table 5. An investigation is carried out to see the performance of this classifier when noise increases.6 show the testing data set with 1%. The different percentage of noise which is 1%.3: The Values of Statistical Parameters 5. 2% and 3% noise respectively.4.5 and 5. The values of these statistical parameters are given in Table 5. Tables 5.2. the ANFIS classified MRI and Iris subjects with the accuracy of 93. The test performance of the ANFIS was determined by the computation of the statistical parameters such as specificity.

03 1600.76 884.84 1191.67 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 1238 1812 934 2404 1584 1177 1751 869 2345 1585 1179 1689 866 2471 1583 1251.88 875.22 953.3 2582.77 1706.68 1203.5: Testing Data Set with 2% Noise .4 1673.7 1657.37 1831.66 1201.36 2392.66 2453.74 1849.64 Table 5.4: Testing Data Set with 1% Noise 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 1238 1812 934 2404 1584 1177 1751 869 2345 1585 1179 1689 866 2471 1583 1263.82 Table 5.88 1679.54 1723.11 944.5 878.33 2429.76 1769.83 1189.52 1787 887.06 1616.44 1663.65 2557.68 2369.

95 2607.52 1215.04 2416.6: Testing Data Set with 3% Noise Figure 5.68 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 1238 1812 934 2404 1584 1177 1751 869 2345 1585 1179 1689 866 2471 1583 1276.31 1740.64 892.49 1213.11 1867.32 1695.33 962.1 1689.99 2477.46 Table 5.5 896.28 1804.5: The misclassified output of ANFIS classifier with 1% noise .09 1632.

6: The misclassified output of ANFIS classifier with 2% noise Figure 5.7: The misclassified output of ANFIS classifier with 3% noise .69 Figure 5.

9.70 Percentage of Noise 1% 2% 3% Number of Subjects 15 15 15 Number of subjects classified correctly 11 8 6 Table 5. By using the SPREAD value of 10 the PNN classifier has misclassified 2 out of 15 subjects. Results from the PNN classifier are all displayed as a summary of the performance on both training and test data. 1237 1811 933 2403 1583 1176 1750 868 2344 1584 1177 1688 865 2470 1582 Input Vector 1238 1239 1812 1813 934 935 2404 2405 1584 1585 1177 1178 1751 1752 869 870 2345 2346 1585 1648 1179 1180 1689 1690 866 867 2471 2533 1583 1642 Table 5.4 Comparisons of Result Probabilistic Neural Network (PNN) is used as the border line or benchmark for this project in investigating the classification of data. Performance from the PNN and ANFIS method is compared and it is summarized in Table 5.7: The ANFIS Test Results with Noise Consideration 5.8: PNN Target Results Target 1 2 3 4 5 1 2 3 4 5 1 5 3 1 5 .

8: Chart Comparison .67% ANFIS 15 1 93.71 Methods Application Data Overall Misclassified Performance Rating PNN 15 2 86.33 Overall Misclassified Figure 5.33% Table 5.9: Comparison between PNN and ANFIS Classification Results 100 86.67 80 60 40 20 0 2 PNN 1 ANFIS Performance Rating 93.

Unfortunately. .CHAPTER 6 CONCLUSIONS AND FUTURE WORKS 6. The combination of both. neural network and fuzzy logic or known as neuro fuzzy model has been tested and evaluated for classification and detection of MR images. currently there is no classification strategy that can accommodate all its applications. image classification has many forms and uses.1 Conclusions Medical image classification is an important thing in medicine. Fuzzy logic enables us to use the uncertainty in the classifier design and consequently increase the credibility of the system output. Whether it is for diagnostic purposes or practically applied in image guided surgeries. The system that has been designed is used for classifying and detecting tumors in brain MR images. It allows for biological structures to be isolated non-invasively. The system involves two major modules. Fuzzy set theory plays an important role in dealing with uncertainty when making decisions in medical applications. one that performs classification and the other one that detects the tumors from the images. Neuro fuzzy is suitable and feasible to be used as supportive tools for medical problems. The proposed technique involves training the ANFIS classifier for symptoms detection.

Design a Sugeno FIS appropriate to the classification problem. Test the results using the testing data. given actual input classification data. the following steps need to be performed: 1. any of several optimization routines could be applied to adjust parameters that will reduce some error measure (usually defined by the sum of the squared differences between actual and desired response). in order to compute the membership function parameters that best allow the associated fuzzy inference system to track the given input/output data. ANFIS constructs a fuzzy inference system (FIS) whose membership function parameters are tuned (adjusted) using either a backpropagation algorithm alone. The technique that has been proposed here has shown a very encouraging level of performance for the problem of tumor classification and detection in brain . These techniques provide a method for the fuzzy modeling procedure to learn information about a data set. Using a given input/output data set. The computation of these parameters (or their adjustment) is facilitated by a gradient vector. This allows the fuzzy systems to learn from the modeled data. 2.73 ANFIS is the fuzzy logic based paradigm that grasps the learning abilities of ANN to enhance the intelligent system’s performance using a priori Knowledge. Run the ANFIS algorithm on the training data. This learning method works similarly to that of neural network. or in combination with a least squares type of method. The training and testing matrices will be composed of inputs and the desired classification corresponding to those inputs. 5. 3. which provides a measure of how well the fuzzy inference system is modeling the input/output data for a given set of parameters. 4. The parameters associated with the membership functions will change through the learning process. Once the gradient vector is obtained. To use ANFIS in the classification problem. Set up training and testing matrices. Hands optimize the FIS.

Based on the result in Chapter 5.2 Future Work To the fields of MR images classification. There could be many valuable resources on the island. this thesis is like an explorer who has just landed on a big tropical island. especially in the medical problems or field such as lung cancer. it is very crucial to determine the accuracy and precision of the output of the analysis. Nevertheless. it can be concluded that this project has achieved the objectives and problem statements as set earlier in Chapter 2. efficient in classification and easy in implementation. . Efforts were made to reduce the amount of a priori knowledge used. There is a need to apply the classification and detection techniques in many fields especially in the medical field whereby the data is large and time consuming to sort or classify manually. He is only able to gather just a few items on the island to bring back with him because his ship is not big. This makes the approach worth serious consideration for further development as an automatic tool for image classification in medicine. breast cancer and etc. Furthermore. As an overall conclusion. Experimental result indicates that the technique is workable with an accuracy greater than 90%. based on those resources he can see. so as to keep the method as generic as possible. this project is successful as it met the objectives of the project and successfully developed. This technique is fast in execution. 6. the explorer has a strong feeling that the island has tremendous development potential.74 MR images. but the explorer is not able to find out about them because his stay on the island is constrained by time. run and optimized the performance of the classification technique.

• Upgrade the system to the application for different size brain medical images slice as shown in Figure 6.75 This thesis has only explored a small territory in using the neuro fuzzy approach in medical image classification. Modification can be done in terms of ANFIS architecture. . convergences and determination of premise or consequent parameters. learning and training algorithm. By increasing the number of image. more training data set can be performed. not only the images are important. which include the following: • In this project the number of medical image is less. • Get the extra information that could be very useful in detecting some patterns in developing a certain disease. but also some patient information. There are many possible directions for future work. • The use of modified version of ANFIS model instead of Sugeno type ANFIS model. In such a system. the same region of the tumor but in the different slice of brain medical image can be determined. For future work the number of brain medical images should be increased and using the real ones. • Build the complete GUI to interface all the process starting from region of interest to ANFIS classification and detection. By doing this.1. The classification would be better performed after the segmentation method and the segmentation results will be used in classification. Figure 6.1: Different slice of brain medical image • Perform the segmentation process to detect a tumor and also its size.

The method can be useful in the diagnostic of diseases if the possibility of a feature is known in the Region of Interest (ROI) of an image. A combined feature is more meaningful than the one feature representation alone and can significantly improve the performance.2 gives the block diagram of the technique. Figure 6. tumor shape and size features.76 • Design a combining multiple classifiers that use different features such as texture features.2: Block Diagram of the Neuro Fuzzy Approach . Figure 6. etc in order to get an accurate classification of the normal and abnormal brain image. and the images can be classified into category of diseases.

A. 23(3). vol. 121-128. 2006. 665685.77 REFERENCES 1.” Computers and Biomedical Research. Virant-Klun. 1993. A. Kuncheva. 16. Vol. IEEE Proceedings of the Geometric Modeling and Imaging.R. 305321. Oweis and Muna J.” Artificial Intelligence in Medicine. Steimann.Y. Journal of Electrical Engineering. Aribi. 1996-2008. pp. 9.J. 2000. Rami J. S. “ANFIS: Adaptive-network-based fuzzy inference system. 32. Nevill. Ramiro Castellanos and Sunanda Mitra. Cambridge .Benamrane. 1999. L. Joseph P.5-6. 3. 8. Jang. Spencer. Virant.-S. “Fuzzy diagnosis. . vol. The Basics of MRI. 2002. Hornak. 2. D.F. 7. 6.56. J. and Cybernetics. Taktak. A.” IEEE Transactions on Systems.: Foundations of Neural Networks. No. 4. “Fuzzy logic alternative for analysis in the biomedical sciences. N. 24. IEEE Symposium on Computer-Based Medical Systems. Kasabov. Bevan. pp. 149-165. MIT Press. vol. 1999.” Artificial Intelligence in Medicine.A. 5. N. I. Fuzzy Systems and Knowledge Engineering.G.I. Man. L. “A Combined Neuro Fuzzy Approach for Classifying Image Pixels in Medical Applications”. 2005. J. S.Kraoula. vol.Sunna’. Roden. “Automatic detection of distorted plethysmogram pulses in neonates and paediatric patients using an adaptive-network-based fuzzy inference system. pp. Belal. F. S. “Fuzzy Neural Networks and Genetic Algorithms for Medical Images Interpretation”. “Segmentation of magnetic resonance images using a neuro-fuzzy algorithm”.1996. pp.

Y. Artificial Intelligence in Medicine. C. Rucci. 16. Journal of Magnetic Resonance Imaging. F.L. Panella. K.F.M. 31:73–89. 239-240. G. Jin Young Choi. 2007. IEEE International Conference on Systems.Fuzzy and Soft Computing. pp. Neural Network Segmentation of Magnetic Resonance Spin Echo Images of the Brain.. G. 1996. Brain tumor classification based on long echo proton MRS signals. 17. vol. I. Input selection for ANFIS learning. 13. Caramella. et al. 1993.355-362. 12. Neuro-Fuzzy and Soft Computing-A computational approach to learning and machine intelligence. p.. Han Witjes et al. Chin-Ming Hong et al.Valli. Jin Hee Na. Myoung Soo Park. 19. Jang. F. Meneganti.Frattale Mascioli. Jang. 2004. IEEE International Joint Conference on Neural Networks.Martinelli. . Man and Cybernetics.. 1994. IEEE Transactions on Neural Networks. Signal Processing.. Fuzzy Systems. 20. Martinelli.Inc. Sun. 11. 22. V012: 1493 -1499.-S. A constructive approach to neurofuzzy networks. pp. Jang JSR. Pearson Education Inc. “A Novel and Efficient Neuro-Fuzzy Classifier for Medical Diagnosis”. 21.R. USA. Fuzzy Neural Network for Classification and Detection of Anomalies. “Principal Component Neural Networks: Theory and Applications”. Coppini. Fuzzy model identification based on cluster estimation.-T. 1998. 18. Kung. New Jersey: Prentice Hall. 2003. Proceedings of the Fifth IEEE International Conference. J. V 15.341-345. R. no 5 September 1998. and E. 2006. Saviello y Tagliaferri. Mizutani .78 10... 1997. 64 : 347-358. Diamantaras and S. Cagnoni. J.M.S. 15. John Wiley & Sons. 2 (3): 267-278. M. 2001. Journal of Biomedical Engineering. 2005. ANFIS synthesis by hyperplane clustering.9. A.Chiu. Multispectral magnetic resonance image analysis using principal component and linear discriminant analysis. IFSA World Congress and 20th NAFIPS International Conference. Journal of Intelligent and Fuzzy Systems. 1996. 17:261–269. Lukas L.1: 340 -345. S. Mascioli.Vol. Neuro. Sun C T and Muzutani E. 14. "PCA-based feature extraction using class information". Rizzi.

26. M. 3. Fuzzy toolbox. 1992. Attikiouzel. Matlab Home. 31. 2. B . 892-896. E.. 2(4). 3. Hall. vol. 60-65. 24. 672-682. D. F. vol. Neural Nets. 1992. Turksen . Morrison and Y.. 25. Cambridge University Press. Image Processing Toolbox.” Proc. 27.79 23. R. 2002. Anil Jain. 101-108. Massachusetts.. P. Digital Image Processing. 29. Chichester. Richard. et al. “A Hybrid Fuzzy Neural Networks for the Detection of Tumors in Medical Images”. Valli. R.Benamrane and al. ” Fuzzy Sets Sys. analysis and algorithm development: http://www. 130. Vol. 0. American Journal of Applied Sciences.. Image Processing. Fundamentals of Digital Image Processing. User manual. Vol. Baltimore. 1998.mathworks. C. L. Wiley. Poli. Journal of Computing and Information Technology.. Gonzalez. 33. “ A fuzzy neural network for pattern classification and feature selection .. Neural Networks and Prior Knowledge Help the Segmentation of Medical Images. 1997. N. Li. Addison Wesley. Klawonn. M.. . pp. Cagnoni and Coppini. pp. 2005. “A comparison of neural network and fuzzy clustering techniques in segmenting magnetic resonance images of the brain. 30. Rafael. Mathworks Inc..6 No. “A probabilistic neural network based image segmentation network for magnetic resonance images. 1999. Foundations of Neuro-Fuzzy Systems. Paul Suetens. Nauck. Math works worldwide.” IEEE Bans. p. 28. and Kruse. Woods. England. Fundamentals of Medical Imaging. Mukaidono and I . London. Neural Nets. 32. Conf.com/products/image. 2002. pp. Prentice Hall.1989.

. al = imread(imagname1). %location point bigger than 20 .[64 64]). %Fills holes in the binary image BW figure (2).graythresh(I2gray)). %Resize an image to 64x64 pixel b3 = rgb2gray(b2).'s').imshow(Ifill).bwarea(Ifill).'hole'). %Convert an image to a binary image Ifill = imfill(Ibw. %substract two binary image if X <=10 %if size of input binary image match with prior image c = a3-b3.title('original') I2gray = rgb2gray(b1). %Resize an image to 64x64 pixel a3 = rgb2gray(a2).ext):'. %find differential between image a and b d = c>20.graythresh(Igray)).'hole'). Ibw = im2bw(Igray.. %convert an RGB image or colormap to grayscale imagname2 = input('\n Enter the name of the input image file (filename.title('abnormal') X = bwarea(I2fill) . a2 = imresize(al. %Fills holes in the binary image BW figure (1). %find the point bigger than 20 fprintf('\n Location of abnormal tissues.imshow(I2fill). b1 = imread(imagname2).'s').') imagname1 = input('\n Enter the name of the prior image file (filename.j]=find(d).[64 64]). %Convert an image to a binary image I2fill = imfill(I2bw.') [i.ext):'. b2 = imresize(b1. I2bw = im2bw(I2gray.80 APPENDIX A MATLAB source code for Transformation image to Data %Entering the image for MATLAB fprintf('\n Entering the image for MATLAB.. %Convert an RGB image or colormap to grayscale %Test size of priori model and input model Igray = rgb2gray(al)..

fprintf('\n Saving. for k=1:3 B=0.'s'). if (sav == 'y') fprintf('\n You choose to SAVE the Result') name = input('\n Type the name of the new file (filename) : '.81 % Find the coordinate T=[ ]. B=B+1. C=C+64. fprintf('\n The new file is called %s and it is saved in MATLAB working Directory'.name) else fprintf('\n You choose NOT to SAVE the Result') end .'ans'). end fprintf('\n Coordinate of damage tissues:'). end end T=[T L(k)].') save(name.. C=0.. for A=j(k) while A~=B L(k)=i(k)+C.'s'). T %Saving the output sav = input('\n Do you like to SAVE Result? (y/n) : '.

82 %If size of input Dot match with prior image else fprint('\n Size of input image not match with size of prior image, Please Try again!!') return end

83

APPENDIX B

PCA Algorithm code

% PCA-based brain mri recognition system

% See also: CREATEDATABASE, EIGENMRICORE, TUMORRECOGNITION

clear all clc

% Initial directory paths can be customized and fixed TrainDatabasePath = uigetdir('D:\Program Files\MATLAB\R2006a\work', 'Select training database path' ); TestDatabasePath = uigetdir('D:\Program Files\MATLAB\R2006a\work', 'Select test database path');

prompt = {'Enter test image name (a number between 1 to 10):'}; dlg_title = 'Input of PCA-Based Brain MRI Recognition System'; num_lines= 1; def = {'1'};

TestImage = inputdlg(prompt,dlg_title,num_lines,def); TestImage = strcat(TestDatabasePath,'\',char(TestImage),'.jpg'); im = imread(TestImage);

T = CreateDatabase(TrainDatabasePath); [m, A, EigenMRI] = EigenMRICore(T); OutputName = TumorRecognition(TestImage, m, A, EigenMRI);

SelectedImage = strcat(TrainDatabasePath,'\',OutputName);

84 SelectedImage = imread(SelectedImage);

imshow(im) title('Test Image'); figure,imshow(SelectedImage); title('Equivalent Image');

str = strcat('Matched image is : ',OutputName); disp(str)

Train_Number = 0.Path of the training database %File management TrainFiles = dir(TrainDatabasePath).name.85 APPENDIX C CreateDatabase function code function T = CreateDatabase(TrainDatabasePath) % Description: This function reshapes all 2D images of the training database % into 1D column vectors.name. Suppose all P images in the training database have the same size of MxN.db')) Train_Number = Train_Number + 1. for i = 1:size(TrainFiles.'.'Thumbs. % % % Argument: % % Returns: % % % % T .')|strcmp (TrainFiles(i).')|strcmp(TrainFiles(i). So the length of 1D column vectors is MN and 'T' will be a MNxP 2D matrix. containing all 1D image vectors.'.name. % Number of all images in the training database end end %Construction of 2D matrix from 1D image vectors T = [ ]. TrainDatabasePath ..1) if not(strcmp(TrainFiles(i). . Then. it puts these 1D column vectors in a row to % construct 2D matrix 'T'.A 2D matrix.

img = imread(str). img = rgb2gray(img).str). str = strcat(TrainDatabasePath. % Reshaping 2D images into 1D image vectors T = [T temp]. str = strcat('\'.irow*icol.jpg'). However.1). [irow icol] = size(img).str. temp = reshape(img'.'. % 'T' grows after each turn End . it is not mandatory! str = int2str(i).86 for i = 1 : Train_Number % Chose the name of each image in databases as a corresponding number.

for i = 1 : Train_Number temp = double(T(:.(M*NxP) Matrix of centered image vectors %Calculating the deviation of each image from mean image A = [ ]. Suppose all P images in the training database have the same size of MxN.A 2D matrix.(M*Nx1) Mean of the training database T .(M*Nx(P-1)) Eigen vectors of the covariance matrix of the training database . % Computing the average MRI image m = (1/P)*sum(Tj's) Train_Number = size(T. containing all training % image vectors and returns 3 outputs which are extracted from training % database. EigenMRI] = EigenMRICore(T) % Description: This function gets a 2D matrix. A m . A.i)) .2).m. containing all 1D image vectors. % Merging all centered images end .2). So the length of 1D column vectors is M*N and 'T' will be a MNxP 2D matrix. % % Argument: % % % % % % Returns: % % % % %Calculating the mean image m = mean(T. EigenMRI .87 APPENDIX D EigenMRICore function code function [m. A = [A temp].

i)]. end end %Calculating the eigenvectors of covariance matrix 'C' EigenMRI = A * L_eig_vec.2) if( D(i. %Sorting and eliminating eigenvalues L_eig_vec = [ ].88 %Snapshot method of EigenMRI methods L = A'*A. % L is the surrogate of covariance matrix C=A*A'. % Diagonal elements of D are the eigenvalues % for both L=A'*A and C=A*A'.i)>1 ) L_eig_vec = [L_eig_vec V(:. % A: centered image vectors . [V D] = eig(L). for i = 1 : size(V.

A. TestImage .(M*Nx1) Mean of the training database. which is output of 'EigenMRICore' function.(M*Nx(P-1)) Eigen vectors of the covariance matrix of the training database. % % Returns: OutputName .Name of the recognized image in the training database. EigenMRI) % Description: This function compares two brain images by % projecting the images into imagespace and % measuring the Euclidean distance between them. which is output of 'EigenMRICore' function. % % Argument: % % % m . % % % A . m.Path of the input test image %Projecting centered image vectors into imagespace ProjectedImages = [ ]. % Projection of centered images into imagespace .i). for i = 1 : Train_Number temp = EigenMRI'*A(:.2).(M*NxP) Matrix of centered image vectors. which is output of 'EigenMRICore' function. Train_Number = size(EigenMRI.89 APPENDIX E TumorRecognition function code function OutputName = TumorRecognition(TestImage. % % % % EigenMRI .

i). end [Euc_dist_min. InImage = reshape(temp'.irow*icol.1).1). Euc_dist = [Euc_dist temp]. for i = 1 : Train_Number q = ProjectedImages(:. temp = ( norm( ProjectedTestImage . Difference = double(InImage)-m. % Centered test image ProjectedTestImage = EigenMRI'*Difference.'.90 ProjectedImages = [ProjectedImages temp]. OutputName = strcat(int2str(Recognized_index). end %Extracting the PCA features from test image InputImage = imread(TestImage). [irow icol] = size(temp).Recognized_index] = min(Euc_dist).q ) )^2. .:. temp = InputImage(:.jpg'). % Test image feature vector %Calculating Euclidean distances Euc_dist = [ ].

numMFs=input('\nNumber of membership function: ').'s'). axus2=strcat(name_outputdata. without neither number nor extension): '. epoch=input('\nNumber of epoch for training process: '). % Loading Training data loadingtraining=load(axus). subplot(211).extension). % strcat combine two strings together axus=strcat(name_trainingdata.dat'.91 APPENDIX F Neuro Fuzzy Algorithm code % Classification using ANFIS % Input Phase clear name_trainingdata=input('\nEnter name of Training Data files (only prefix. outmftype=('linear').extension). inmftype=str2mat('gbellmf'.'pimf'). without neither number nor extension): '. extension='. axus1=strcat(name_checkingdata. bar(data). title('Training data'). . bar(loadingchecking).'s').'trimf'. loadingoutput=load(axus2). data=[loadingtraining loadingoutput].extension). figure(1). subplot(212). name_checkingdata=input('\nEnter name of Checking Data file (only prefix. name_outputdata=input('\nEnter name of Output file (only prefix. without neither number nor extension): '. loadingchecking=load(axus1).'s').

subplot(312).'input'. % Classification/Fuzzy Inference System Phase to get the FIS fis=genfis1(data.1). legend('input1'.'input'. plotmf(fis.epoch. subplot(311).1) .3).loadingchecking).3).checkfis.inmftype.'output'). time=toc. subplot(312).'input2'. plotmf(fis. % Anfis Model Training Phase % Recording time for ANFIS training tic.'input'. Total_time_for_ANFIS_training=time fprintf('\n<<<<<<<<<<<<<<<<<<<<WARNING!!!!>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>') fprintf('\nResults above show second column is training error') fprintf('\nand third column is checking error') fprintf('\n<<<<<<<<<<<<<<<<<<<<WARNING!!!!>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>') fprintf('\n') % Display Error for Training and Checking final_training_error=errortraining(epoch.'input3'. plotmf(fis.'input'.stepssize.[1 1 1 1].1).92 title('Checking data').2). [fistraining. subplot(313).outmftype) figure(2).2).errorchecking]=anfis(data.'input'.'input'. plotmf(fistraining. plotmf(fistraining. figure(3) subplot(311).errortraining.fis.numMFs. plotmf(fistraining. subplot(313).

fistraining). output=round(out) % Detection Phase target=load(axus3).extension).d]=size(target). applicationdata=strcat(name_applicationdata. else saving==2.'s'). number=0.93 final_checking_error=errorchecking(epoch. end.1) maximum_training_error=max(errortraining). end for i=1:size(output) . average_checking_error=(maximum_checking_error+minimum_checking_error)/2 average_global_error=(average_training_error+average_checking_error)/2 % Save the trained FIS saving=input('\nSave the Trained FIS. eg: yes=1 or no=0?= '). % Application Phase %Evaluate the fis based on new set of data(application data) name_applicationdata=input('\nEnter name of Application Data files (only prefix. [c.b]=size(output). loading=load(applicationdata). minimum_training_error=min(errortraining).without neither number nor extension): '. if saving==1.extension).'s'). if a~=c error('Mismatched input and output data matrices'). writefis(fistraining).without neither number nor extension): '. minimum_checking_error=min(errorchecking). axus3=strcat(expected_result. [a. expected_result=input('\nEnter name of Expected Data Result files (only prefix. average_training_error=(maximum_training_error+minimum_training_error)/2 maximum_checking_error=max(errorchecking). out=evalfis(loading.

94 if output(i. disp('InCorrect') number=number+1. end end Total_Misclassified=number .1)~=target(i. disp('Correct') elseif output(i.1).1)==target(i.1).

95 APPENDIX G Brain MR Image for Class 1 MRI1-1 MRI1-2 MRI1-3 MRI1-4 .

96 APPENDIX H Brain MR Image for Class 2 MRI2-1 MRI2-2 MRI2-3 MRI2-4 .

97 APPENDIX I Brain MR Image for Class 3 MRI3-1 MRI3-2 MRI3-3 MRI3-4 .

98 APPENDIX J Brain MR Image for Class 4 MRI4-1 MRI4-2 MRI4-3 MRI4-4 .

99 APPENDIX K Brain MR Image for Class 5 MRI5-1 MRI5-2 MRI5-3 MRI5-4 .

Sign up to vote on this title
UsefulNot useful