Nonlinear System Identification Based on Evolutionary Fuzzy Modeling ‘Toshiharu Hatanaka Yoshio Kawaguchi and Katsuji Uosaki Department of Information and Physical Science, (Osaka University Suita, $65-0871, Japan Enmail: hatanaka@istosaka-v.ac ip Abriract—The local modeling such as TSK fuzzy modeling 4s well known as a practical approach for nonlinear system ‘modeling. Tn this approach, a selection of membership functions ‘makes much effec upon the model performanee. Is usually ‘determined by the expert's knowledge for the objective systems. Hlomever, tis often dificult to give appropriate membership functions for unknown complex dynamical system without any prior Information. In this paper, we deal with the approach fo give appropriate fuzzy membership functions based on the observed input and output data using genetic algorithm. Then, fan application to identification of nonlinear systems is considered andthe availablity ofthe proposed method is ustrated by some ‘numerical examples. I. INTRODUCTION Black box modeling of unknown systems based on the ‘observed input and output data is called as system identifi. ‘cation [1]. Tt has an important role in a lot of engineering problems such as signal processing, time series prediction, ‘control system design, fault detection and diagnosis etc. Sys tem identification algorithms were developed for the linear system modeling. Then, nonlinear system identification has been much studied in the last two decades and many ap- roaches have been proposed such as Volterra series, artificial neural networks and fuzzy models (2). On the other hand the complenity of engineering systems has been increasing, it is required for control engineer 0 develop the practical approaches dealing with such systems. Generally in order to deal with such complex systems, itis necessary to construct ‘4 model representing the system globally. However, it is very difficult to make a global optimum model applicable to full range of operation of the objective system because such complex systems are usually composed by a huge ‘umber of components relating with each other and have wide range of operation. Hence, an idea of local modeling has received much attention in modeling of such complex systems (3], [4], [5], (6) In particular, fuzzy models such as TSK fuzzy model are useful fools for modeling such complex systems, since they are characterized by fuzzy partitioning of the operating range Z ofthe systems and a simple local model fi within each sub-range Z; [7}.(8]. The relative validity of the local model fj is expressed by the membership function ‘Ha, for the fuzzy subset Z). Each local model has adequate accuracy for the corresponding sub-range Z;. The quality of the overall TSK fuzzy model is heavily dependent on the (0-7803-8515-2/08/520.00 ©2004 IEEE. fuzzy panttioning. In other words, the primary important steps in system identification of such fuzzy model are how the ‘optimal fuzzy partition is performed and how the necessary variables are selected. From this point of view, automatic fuzzy partitioning algorithm based on the observed input and output ‘data using genetic algorithm is proposed in this paper. We also represent that the proposed approach should be able to deal with time varying system by small modifications. Then, effectiveness of the proposed approach is shown by some ‘numerical examples. perry eam” = om Rs Fe. Loca modeling TI, Fuzzy MopeLinc Let's consider the identification problem of @ nonlinear dynamical system represented by the following nontinear autoregressive models with exogenous input (NARX model): 910) = Fle = Door 9lt—m) M(t Dy yale na) + eC0) = flat) +e(0) ® = Da t= m)s uty) ule =m)? @ Here u(t) € R and y(¢) € R ase the system input and output, respectively. e(¢) indicates an observation noise independently ‘dentcally normally distributed with zero mean and unknown variance, and 9(t) is called an information vector composed by the input and output data. ny and ng are unknown model orders. 046‘Then, we consider here the TSK fuzzy modell7). It decom. poses the global model into several local models. A. local model has validity only within a certain operating range Z; and itis assumed to have simple linear structure such as ARX model or the like. An operating range Z),(1 = 1,---,N) ean be considered as a fuzzy subset which is determined with membership function iz(@). Then, we assume that each local model can be expressed by (linea) ARX model (see Fig2), the TSK fuzzy model consists ofa set of fuzzy rules, If a(t) € 2; Then W(a)o(t) + Bilal) = ayy 1) +--+ anal — My) + bo H(t) +4 bay =a) wd ° a! is delay operator, q7!u(1) = u(t — 1). 2{1) is a veotor fof premise variables derived from information vector $40), ‘The orders of the polynomials Ai(g) and Bi(q). n, and ny are assumed 10 be known. The regime is defined by fuzzy sets Z; and the consequent is a local ARX model. Using the eft), ean ‘TSK fuzzy model, the output of the fuzzy system is given by following equations based on fuzzy inference. y wilt) ale) Daal) ol wo o Here, » indicates interpolation weights which are determined by the fuzzy membership functions. As shown in (4) and (5), the fuzzy paritioning (.e. membership function) obviously affects on the model performance. It is required that suitable fuzzy partitioning should be considered to give a good rhode! for whole objective system. MM, EvouTionay Fuzzy MODELING Here, we assume that fuzzy partitioning performed based con the observed input and output variable at time instance ¢ and we consider the trapezoidal function as a membership function for input x(¢) and y(e) respectively. For example, Figure 3 shows the partitioning inthe case of both membership functions having (wo operating range. wt) u(t) Fig 5. Two-cimensiona ferry paroning fey prong Each ARX-model is assumed second order system for sim- plicity and then TSK fuzzy model can be written as following Rules; : If u(t) is A; and y(0) is By Then yy 1) =ayav) +ayaytt— 1) F by,au(t) + by2ult~ 1) ) Here, A; and By indicate the operation range defined by membership functions 4,(2) and By(), respectively. The in- ‘erpolation weights are decided by fuzzy inference as follows may = mi Ai(u(e)), By) @ ‘Then, the estimated system output at time instance ¢ becomes ZL midi. 3 ® Now, an evolutionary fearing approach to determination of fuzzy membership function based on the observed input and output data i considered ‘The membership function set can be diséely encoded to the chromosome in real coded genetic algorithm as shown in Fig. yee and Ymuz con be determined by the observed ata set. The 0 tapeaoidal membership functions for output ae defined by the 2(n~ 1) parameters {p1sP2>~-sP2j-1)} Similarly membership functions for input ate defined by 2(n— 1) parameters {q1,92+°-* -d2(a.1)}- The chromosome is composed by aligning these 4(n~ 1) parameters serially Then, the fitness function should be evaluated based on its phenotype performance, that is accuracy oF quality of the tlobal system model usually estimated by the mean square 647u(t) qa: 4 Yond + phenotype { genotype AAA elela) Fig. 4 Fazty membership (encions and te eheomocome represenion terror. Prom this viewpoint we define the fitness function as following equation, o where fi indicates fitness value of th individual. MSE; indicates the mean square error of the model given by Eth individual, 1 MSE: = 5 DOW) - HO? 20) where fit) is predicted output of y(t} by estimated model siven by the és individual based onthe past observed input and output data. MSEnge shows the maximum MSE value in ‘he population. Actually the parameters of each ARX mode should be identified by the least square approach with input and output data, which are included in the comesponding ‘operating range defined by cach membership function given by each chromosome Then, we employ the following genetic operations to decide the membership functions using genetic code piven in Fig. + exossover: one-point crossover withthe probabil + mutation : mutation operator acts on randomly selected Tocus and ehange its allele by ay Here, j denotes the randomly selected locus, x) denotes gene at j-th locus and r is random variable sampling from uniform distribution. + Selection + the roulette wheel selection with elitism is employed. + sort: gene should be sorted ascending to avoid discrep- ancy of phenotype of membership function, ayauyte ‘The proposed algorithm is summarized in the flow chart (Fig). T Se ES a Team Soon roy Fig 5. Finzy pattionngalortim with GA, TV, NUMERICAL EXAMPLES Numerical simulation studies have been carried out to verily applicability of the proposed approach to nonlinear system identification A. Example } The frst simulation considers the following nonlinear time series model with switching property. 1.59, - 0.7.1 +0344 m1 6, 1.2y,-03y,-1-05u,+ 13.44, 20 ye 4 <0 where e; is white Gaussian noise with mean 0 and unit variance. Under the random input signal, its average shifts from -5 9 +5 slowly as shown in Fig.6, we use 300 set of input-output data. The following settings are applied to genetic algorithm. Evolution process is terminated after 300 Benerations, we obtained following fuzzy models. TF a Ay THEN $1 = 1.505, ~0.70y,-4 40.194, + 1.0841 (12) and TF uy € Ap THEN, §, = 123y, — 0.3394 ~0.46u + L14yy. (13) 6430 100; 200 300 Fig. 6. Sample ajc of inp sgnl TABLET Ga serniNos FoR peAMPLet Trem Size Of population 20 Crossover rate os Mutation rate 1 Generations 300 Number of membership _2 and the identified membership function is shown in Fig?. ‘The comparison of the observations and their predictions is shown in Fig, we can see the goad performance of the identified modes. The convergence behaviors of mean squire exvoe1= $ (2) =H). which indicates the best individual {in the population is shown in Fig.9. These results show that the proposed algorithm has a satisfactory convergence property and has ability t0 give appropriate membership functions and ‘mean square error. Al 0 u(t) Fig 7. The idmted membership faction Fig. 8. Observed ouput and prediction enor 1.32 1.26 1.20| 1.14 0 100 200 ‘300 jeneration Fs. 9. Comergence bebavior of can stutter of the ite individual B. Example 2 ‘Then, we consider the following nonlinear dynamic system, 32) = —0.3y2(0) +40) + L Su), as, where u(f) is input signal defined by u(e) =2.7sin(nt) +1 3sin(3.2a1} +009), 5) and v(t) is random perturbation normally distributed with ‘mean zero and unit variance. The system is modeled by the proposed method using 1000 set of the observed input and ‘output data under the condition of the sampling rate 0.01. ‘The observed input and output data are shown in Fig 10. We cari out simulation run using following design parameters in genetic algorithm. Then, after 300 generations, we obtain ‘TABLE W GA SETTINGS FoK EXAMPLE Crossover rate Mutation rate Generations Number of membership AC for ig 49the system model such as Rulel: If wis Ay and yy is By, Then Jou 053p1 4047-14 0.0lu +0020. Ruled: IF wis Ay and y; is By, Then Yet = OA8y, +051y,-1 +0004 — 0.014 le If uy is Az and ye is By, Then Ioan = OMB), + 0.53911 +0.03t4 — 0.0141 Ruled: If wis Ap and ys is Bs, Then Yn41 = 06), 0.52341 + 0.0Tuy ~0.024,-1 ‘One step prediction error defined by y(¢) ~Setie(t) is shown in Fig.12, it shows the good performance. Here, . Satie") indicates the prediction of y(e) by the elite individual. Figure II shows the identified membership functions for input w; and output y., respectively. Compare them with Fig.10, these ‘membership functions partition the area appropriately. Fig 10 Operating ange ©. Example 3 In this example, we consider the problem of noalinear time varying system idenifiation. To deal with such time varying systems, a shifting data window is introduced. The window shifts half size ofits width with every L generations of GA and the fitness is evaluated inthis data window only. Then, we consider the following nonlinear system which parameters jump a time instant T +, 307 +y+15u(¢< 7) as -2.0;2-4y+2.0u,(¢2 7) Since the objective system changes tthe time instance 7", the membership functions willbe expected to change ater T*, Using the same input signal and the same sampling rate as Example 2, numerical simulations are carried out under the conditions listed in the TABLE UL The fitness function is evaluated by following modified MSE instead of equation (10), 14 B > ble)- MSE\(R) ny? a7) A ost = 47 \ Ae oa| HE a 3 0.1210.348 2 3 4 ult) Bi os} 8 Be oa 0 1 200203 3 4 yt) Fig. 1. The member tons gen by he ete ni sn oo a oa a a! Fig. 1. One sp retin enor of dente ode proved y ee ind D, k= Ful=01, ‘Where, D is data window size and 1 denotes the time index which gains at every L generation. The example of identified ‘membership functions of both input and output before system ‘change is presented in Fig.13 and ones after system change are presented in Fig. 14, The system models given at 1500"* ‘generation (before system change) and 3000" generation (after system change) are Rulel: If u; is Ay and y, is By, Then as) Sees = OLY, $0.58). 40.09u 000TH 1 Ruled: If w is Az and y, is By, Then Youn = OS1y, + 0.399421 — 0.17 0.2911 and Roulel: If mis Ay and y; is By, Then +0.05u,—0. drat = 0.493, + 0.489) Bu Rle2: If wis Ay and yr is Bo, Then Yi41 = 0.673 +0.30y1-1 ~ 0.164 + 0.1341 650‘TABLE Tem Configuration “Size of populaion 100. Crossover rate as ‘Mutation rate 005 Number of membership 4(2 for input and 2 for output) Data window size 256 L 80 "Time inverval Te Da 57 r 40.96 ay is Az and y is By, Then dest = 05834 40.3891.) +0.25u, 0.144)» Ruled: If is Az and y, is Bz, Then 565+ 0.433444 0.174 = 0.121 ‘respectively, No model build for range defined by membership function B; of 1500” generation because there are n0 data in this range within its data windows. Figure 15 indicates the trajectory of the best fitness in cach generation. We can see the appropriate performance of the model given by the elite individual. By these results, it is shown that the proposed approach has an applicability t identification of time varying nonlinear system, , @ 8 @ 68 6 ™ Fe the membership fenton before system change. V. Conctusions In this paper, we have proposed an evolutionary learning approach to automatic fuzzy modeling of nonlinear dynamical systems. The fuzzy membership functions which are repre- sented by chromosome with real coded gene are determined based on only observed input and output data by genetic algorithm without using any prior knowledge conceming ob- jective system. Numerical simulation results have shown the spplicability of the proposed approach. Then, application 10 nonlinear time varying system modeling was discussed by a ‘numerical example. I was shown that the proposed approach 6st dl h on] | iN ae hid a Fig 15, Tajectory of MSE ofthe best individual has an adaptability for system change, due 0 propery of ‘evolutionary learning, REFERENCES (01 L, Lang : Site teen: Theo forthe User, 2d Eto}. Frente Hal, 199, (2) Olver Nats: Nonlinear Sjoten Idetifcaton, Spige, 200 [5] Re Moray-Smih and. A Jehansen(Eds, Melle Model Approaches 2 Nonlinear Modting and Con, ast Panis. 1987 (4) 7A, Fohncen aod A. Foon “Laval modeling bea oo fr sem ernpveal and semi-rechanistie process" Newel Nervrks or Chemical Engineer, pp. 297-394, 1995, (5) TA font, "Idetieation of nooner systems weg epiial data and 9 pion knowledge ~ an optimisation approach.” Aamarca, Vo132, pp. 137-386, 1936. (6) TA. Bobasea and B.A. Poss, “Operating cegine based proces modeling ad ideotieation” Computes and Chemical Engineering. NGI2L, pp 1594176, 197. (0) TTaagl and Mt. Sapeno, “Farzy idenicaion of systems and ts ppm to modeling and coowol” IEEE Trons. Stews, Man ond Giermetes VIS, ppt 132, 1985. mT fntraka, Ke Unis and. Manabe, “Stee identification in TalaphSigeno foxy madeling” Proc of The 2002 IEEE World Congress on Computational Ineligene, 2002 (91 K-Uoakt'T Hataaks and N"Mnake, “Astomtic operating regime selection it leat modeling” Proof he 15 TEAC Mord Congreso Ausomatie Cone. 2002 (Wo) atanats K. Vesa ad M. Whi, “Optimal Fuzzy Parttonng in Vocal Modeling tas on Evlaionar) Campuation Proc of the 4 ‘Aa-Pase Conference om Sinaated Evolution Sind Learning, 99 340- Seo, 3002