5 views

Uploaded by cicimonaa_viva

save

You are on page 1of 5

**Nonlinear Systems with Probabilistic Uncertainties
**

A. Jamali

Department of Mechanical

engineering, Faculty of Engineering,

The University of Guilan, Rasht, Iran

ali.jamali@guilan.ac.ir

N. Nariman-zadeh, H. Ashraf

Department of Mechanical

engineering, Faculty of Engineering,

The University of Guilan, Rasht, Iran

nnzadeh@guilan.ac.ir

Z. Jamali

Department of Civil engineering,

Faculty of Engineering, The

University of Guilan, Rasht, Iran

zahrajamali_civil86@yahoo.com

Abstract—In this paper, multi-objective evolutionary Pareto

optimal design of Adaptive Neuro-Fuzzy Inference System

(ANFIS) have been used for modeling of nonlinear systems using

input-output data sets with probabilistic uncertainties. In this

way, A Monte Carlo Simulation (MCS) is first performed to

generate input-output data set using some probabilistic

distributions. Multi-objective uniform-diversity genetic

algorithms (MUGA) are then used for Pareto optimization of

ANFIS networks. The important conflicting objectives of ANFIS

networks that are considered in this work are, namely, the mean

and variance of both Training Error (TE) and Prediction Error

(PE) of such ANFIS models. It is shown that a robust ANFIS can

be simply obtained using a criterion based on four values of

means and variances of both TE and PE. The probabilistic

evolved ANFIS model exhibits much more robustness to the

uncertainties involved within the input-output data sets than that

of the deterministic evolved ANFIS model. It is shown that

ANFIS can be successfully applied for input-output data set with

uncertainties so that a robust model can be compromisingly

obtained from some non-dominated optimum ANFIS models

Keywords-component; ANFIS; Pareto; Robust model; Monte

Carlo.

I. INTRODUCTION

System identification techniques are applied in many fields

in order to model and predict the behaviors of unknown and/or

very complex systems based on given input-output data [1]. In

this way, soft-computing methods, which concern computation

in an imprecise environment, have gained significant attention

[2]. Many research efforts have been expended to use of

evolutionary methods as effective tools of soft-computing

methods for system identification [2-3]. Among this method,

fuzzy rule-based systems have been an active field of research

because of their unique ability to build models based on

experimental data. Lee presented the concept of fuzzy sets,

which deal with uncertain or vague information, and paved the

way for applying them to real and complex tasks [4]. Wang

shows that fuzzy systems could be effectively employed as

universal approximators to perform input-output mapping [5].

Moreover, Nariman-Zadeh et al. recently applied SVD in

combination with a genetic algorithm to optimally design a

ANFIS system for modelling purposes that demonstrated its

superior performance in comparison with previous works [6].

However, a fuzzy model consisting of a large number of IF-

THEN rules to map inputs to outputs is not desired due to the

phenomenon of over fitting, which reduces the generalizing

property of the fuzzy model to predict the unforeseen data.

Similarly, the Takagi-Sugeno-Kang (TSK) type fuzzy models

are widely used for control and modelling because of their high

accuracy and relatively small model size. In the TSK models,

which are also known as neuro-fuzzy systems, the consequents

of the fuzzy rules are explicit functions, usually with linear

relationships, of the input variables rather than fuzzy sets. In

other words, as presented by Hoffmann and Nelles the crisp

linear relation portion of the consequents of a TSK fuzzy rule

describes the underlying model in the local multi-dimensional

region specified in the premise part of that fuzzy rule [7].

Therefore, two types of tuning procedures are required for

proper partitioning of the input space and number of fuzzy

rules, known as structural tuning, and for the parameters of the

consequent parts of the fuzzy rules, known as parametric

tuning. In recent years, different approaches have been adopted

for optimal tuning of such models based on either heuristic

search or fuzzy clustering for the premise part and least squares

for linear parameters in the conclusion of the fuzzy rules.

Genetic algorithms have received a great deal of attention for

the optimal selection of the premise part of TSK type fuzzy

rules in Wang and Yen’s literature [8]. In order to identify the

parameters of the consequents, there have been attempts in the

literature to use SVD as a linear optimization technique. Jang

proposed an equivalent approach to the TSK models as an

Adaptive Neuro-Fuzzy Inference System, ANFIS [9]. In this

model a hybrid learning method is used for tuning parameters

in both antecedents and consequents of embodied TSK-type

fuzzy rules. In all of these researches uncertainties are not

addressed and the modelling process is accomplished

deterministically. In fact, it has been shown that designing

without considering uncertainties generally leads to such a

potentially high risk solution [10, 11]. Therefore, it is so

desirable to find such a robust model in which performance

variation in the presence of uncertainties is not high.

With the aid of ever increasing computational power, there

have been a great amount of research activities in the field of

robust modelling and design devoted to the use of Monte Carlo

simulation [12, 13]. MCS is a direct and simple numerical

method but can be computationally expensive. In this method,

random samples are generated assuming pre-defined

probabilistic distributions for uncertain parameters. The system

300

978-1-61284-922-5/11/$26.00 ©2011 IEEE

is then simulated with each of these randomly generated

samples and both mean and variance of the performance

metrics are then evaluated statistically. In Multi-objective

optimization problems (MOPs), there are several objective or

cost functions (a vector of objectives) to be optimized

(minimized or maximized) simultaneously [14, 15]. These

objectives often conflict with each other so that improving one

of them will deteriorate another. Therefore, there is no single

optimal solution as the best with respect to all the objective

functions. Instead, there is a set of optimal solutions, known as

Pareto optimal solutions or Pareto front for multi-objective

optimization problems. Recently, a multi-objective uniform-

diversity genetic algorithm (MUGA) have been presented in

[16, 17] which will also be used in this work.

In this paper, MUGA in conjunction with MCS are used for

Pareto robust optimal design of an ANFIS network for

modeling and prediction of a complex nonlinear system using

input-output data sets with probabilistic uncertainties. In this

way, A Monte Carlo Simulation (MCS) is first performed to

generate input-output data set using some probabilistic

distributions. MUGA are then used for Pareto optimization of

ANFIS networks. The important conflicting objectives of

ANFIS networks that are considered in this work are, namely,

the mean and variance of both Training Error (TE) and

Prediction Error (PE) of such model. It is shown that a robust

model can be simply obtained using a criterion based on four

values of means and variances of both TE and PE. The

probabilistic evolved ANFIS model exhibits much more

robustness to the uncertainties involved within the input-output

data sets than that of the deterministic evolved ANFIS model.

It is shown that ANFIS networks can be successfully applied

for input-output data set with uncertainties so that a robust

model.

II. STOCHASTIC ANALYSIS

In real experimental engineering practice, there exist a

variety of typical sources of uncertainty which have to be

considered through robust modeling approach [10, 11]. Those

uncertainties include plant parameter variations due to

environmental condition, incomplete knowledge of the

parameters, inaccuracies involved with measuring and

experimental apparatus, and etc. In conventional optimum

design of inductive modeling based on input-output data,

uncertainties are not addressed and the optimum model of the

process is often accomplished deterministically. In fact, it has

been shown that optimization without considering uncertainty

generally leads to non-optimal and potentially high risk

solution. Therefore, it is very desirable to find robust model

whose performance variation in the presence of uncertainties

is not high. Generally, there exist two approaches addressing

the stochastic robustness issue, namely, robust design

optimization (RDO) and reliability-based design optimization

(RBDO) [18]. Both approaches represent non deterministic

optimization formulations in which the probabilistic

uncertainty is incorporated into the stochastic optimal design

process. In RBDO approach, some evaluated reliability

metrics subjected to probabilistic constraints are satisfied so

that the violation of design requirements is minimized. In

RBDO approach, some evaluated reliability metrics subjected

to probabilistic constraints are satisfied so that the violation of

design requirements is minimized. In RDO approach, the

stochastic performance is required to be less sensitive to the

random variation induced by uncertain parameters so that the

performance degradation from ideal deterministic behavior is

minimized.

Let X be a random variable, then the prevailing model for

uncertainties in stochastic randomness is the probability

density function (PDF), ( ) x f

X

or equivalently by the

cumulative distribution function (CDF), ( ) x F

X

, where the

subscript X refers to the random variable. This can be given by

( ) ( ) ( )

∫

∞ −

= ≤ =

x

X X

dx x f x X x F Pr (1)

where Pr(.) is the probability that an event (X x) will

occur. Some statistical moments such as the first and the

second moment, generally known as mean value (also referred

to as expected value) denoted by E(X) and variance denoted

by ( ) X

2

σ , respectively, are the most important ones. They can

also be computed by

( ) ( ) ( )

∫ ∫

∞

∞ −

∞

∞ −

= = dx x f x xdF X E

X X

(2)

and

( ) ( ) ( ) ( )

∫

∞

∞ −

− = dx x f X E x X

X

2

σ (3)

In the case of discrete sampling, these equations can be

readily represented as

( )

∑

=

≅

N

i

i

x

N

X E

1

1

(4)

and

( ) ( ) ( )

∑

=

−

−

≅

N

i

i

X E x

N

X

1

2 2

1

1

σ (5)

where

i

x is the i

th

sample and N is the total number of

samples.

III. MODELLING USING ANFIS

An ANFIS consisting of a set of TSK-type fuzzy IF-THEN

rules can be used in modeling to map inputs to outputs. The

formal definition of such identification problem is to find a

function,

^

f , so that it can be approximately used instead of

the actual one, f . This is done in order to predict output

^

y

for a given input vector { }

n

x x x X ..., , ,

2 1

= as close as

possible to its actual output y . The problem is now to

determine an ANFIS that minimizes the mean square error

(MSE) between the actual and predicted output, that is

301

M

y y

MSE

M

i

i i ∑

=

⎥

⎦

⎤

⎢

⎣

⎡

−

=

1

2

^

(6)

In this way, a set of linguistic TSK-type fuzzy IF-THEN rules

is designed to approximate

^

f by f using M observations of

n-input-single-output data pairs ) ,..., 2 , 1 )( , ( M i y x

i i

= . The

fuzzy rules embodied in such ANFIS models can be

conveniently expressed using the following generic form

l

n

i

i

l

i

j

l n

j

l

j

l l

w x w y THEN A is x

AND A is x AND A is x IF rule

n

0

) (

) (

2

) (

1

2 1

:

∑

=

+ =

(7)

in which { } r j

l

,..., 2 , 1 ∈ and { }

l l

n

l l l

w w w w W

0 2 1

, ,... , ∈ are

the parameter sets of the consequent of each rule. The entire

fuzzy set in x

i

space is given as

{ }

) ( ) 2 ( ) 1 ( ) (

,..., ,

r i

A A A A = (8)

These fuzzy sets are assumed to have a Gaussian shape

defined on the domains [ ] ( ) n i

i i

,..., 2 , 1 , = + − β α In this

way, the domains are appropriately selected so that all the

fuzzy sets are complete, that is, for any [ ]

i i i

x β α + − ∈ ,

there exists

) ( j

A in Eq.(7) such that the degree of

membership function is non-zero. Each fuzzy set

{ } ( ) r j A

j

,..., 2 , 1

) (

= represented by Gaussian membership

functions in the form

( )

⎟

⎟

⎠

⎞

⎜

⎜

⎝

⎛

−

− =

2

2

2

exp

) (

j

j i

A

c x

j

σ

μ (9)

Where

j j

c σ , which optimally select by GAs are adjustable

centers and variances in antecedents, respectively. More

details about modeling of complex system using ANFIS can

be found in [6]. The method are suggested by some author in

[6] is adopted here for optimal design of ANFIS network for

modeling of uncertain input-output data set. Therefore, in this

paper, a hybridization of genetic algorithms and SVD is

applied for the optimal design of ANFIS.

IV. PARETO ROBUST MODELLING OF NONLINEAR SYTEM

The input–output data used in robust modeling involve 100

data pairs randomly generated from a nonlinear system [19]

with three inputs x

1

, x

2

, x

3,

and a single output y given by

5

3

x ,

2

x ,

1

x 1 )

1.5

3

x

1.0

2

x

0.5

1

x (1 y ≤ ≤

−

+

−

+ + = (6)

There are 50 pattern numbers which have been randomly

selected from those data pairs to train such ANFIS model.

However, a testing set which consists of 50 unforeseen input-

output data samples during the training process is merely used

for testing to show the prediction ability of such evolved

ANFIS models during the training process.

There are two main concepts involved within ANFIS

networks design, namely, the parametric and the structural

identification problems. In this way, some of authors present a

hybrid method which using genetic algorithms (GA) and

singular value decomposition (SVD) method to optimally

design the Gaussian membership functions of antecedents and

the vector of linear coefficients of consequents, respectively

[6]. The methodology in these references has been successfully

used in this paper to obtain the robust ANFIS model for above

100-input-1-output data minimizing the mean values and

variances of both training and testing of the input-output data

table simultaneously. In the deterministic approach, the mean

square errors (MSE) of training and prediction errors are

chosen for the bi-objective Pareto optimization approach of

ANFIS models using the methodology explained in detail in

[6]. Figure 1 depicts the Pareto front of both training and

prediction errors of ANFIS models for deterministic input-

output data table. Points A and B stands for ANFIS model

having the least prediction and training errors, respectively.

Design point C, however, depicts the tradeoff in such models

which can be reasonably chosen as an optimal model

compromisingly. The ANFIS’ values of TE and PE of

deterministic optimal design points A, B, and C are given in

Table 1. The good behavior of design point C for modeling and

prediction of input-output data table have been shown in figure

2.

Fig. 1: Pareto front of TE and PE of ANFIS models in

deterministic design

302

Fig2. Comparison between actual output and ANFIS output

of design point C for deterministic data set

Ability of design point C (the best design point for

deterministic data table) for modeling and prediction of

uncertain input-output data set is evaluated by 2500 different

data tables which generated by MCS with 10 percent Gaussian

probabilistic distribution of all three input in the input-output

data set. It is clear from figure 3 for uncertain input-output data

set variances of both TE and PE of design point C is highly

increased. Therefore, modeling of uncertain input-output data

set has to consider through robust modeling approach. In robust

modeling approach, the values of means and variances of both

TE and PE of the ANFIS networks is evaluated based on 2500

different data table which generated by MCS. The multi-

objective Pareto design of ANFIS networks is now used to

obtain some non-dominated ANFIS models using statistical

measures given in equations (4) and (5). Consequently, 119

different non-dominated ANFIS models have been obtained

based on four objective functions of the mean and variance of

both TE and PE. In order to compromise among the solutions

obtained, all the values of these objective functions returned by

the MCS are normalized. The minimum value of the sum of

those normalized values is then simply selected as the robust

ANFIS model. The performance of this design point denoted

by point D is given in Table 1 both for deterministic and

probabilistic approaches using MCS for 2500 data sets. It is

very evident from this table that the stochastic behaviour of the

design point D in terms of mean and variance for both TE and

PE is significantly superior to those obtained in deterministic

approach (design points A, B, and C) and thus exhibits more

robustness in the existence of probabilistic uncertainties. Figure

3 depicts the location of the design point D among other non-

dominated design points in the plane of the variances of both

TE and PE. It is clear that the selection criterion leads to the

design point with almost least variances whose values are

given in Table 1.

Figure 2: Non-dominated design points and the selected one

point D in the plane of variances of errors.

I. CONCLUSION

A multi-objective genetic algorithm was used to optimally

design ANFIS model from a robustness point of view in a

probabilistic approach. The objective functions which often

conflict with each other were appropriately defined using some

probabilistic metrics. The multi-objective optimization of

robust ANFIS models led to the discovering some important

trade-off among those objective functions. The framework of

such hybrid application of multi-objective GAs and Monte

Carlo Simulation of this work for the Pareto optimization of

robust ANFIS model using stochastic objective functions is

very promising and can be generally used in the optimum

design of ANFIS models in real-world complex systems with

probabilistic uncertainties.

Table 1: Training and prediction errors of design points in both deterministic and probabilistic approaches

Design

Point

Deterministic Values of TE

&PE

Probabilistic measures of TE & PE (evaluated by

2500 MCS)

Training

Error

Prediction Error

Mean of

TE

Mean of

PE

Variance

of TE

Variance of

PE

A 0.001003 0.009107 0.01235 0.01894

1.32E-

03

1.98E-03

B 0.002251 0.004359 0.06489 0.04297

1.15E-

04

1.98E-03

C 0.001038 0.005660 0.06248 0.07262 1.98E- 1.03E-04

303

03

D 0.009213 0.005916 0.006818 0.021771

4.41E-

08

8.96E-08

REFERENCES

[1] Astrom, K. J. and Eykhoff, P., "System identification, a survey",

Automatica, 7, 123162, 1971.

[2] Sanchez, E., Shibata T. and Zadeh, L. A., "Genetic Algorithms and

Fuzzy Logic Systems", World Scientific, 1997.

[3] Nariman-zadeh, N., Marzbanrad J. and Jamali, A., "Stiffness modelling

of rubber engine mounts by poly-nomial neural networks and genetic

algorithms" Proc. ESM2004, France, 181187, 2004.

[4] Lee, C. C., "Fuzzy logic in control systems: Fuzzy logic controller",

IEEE Trans. Sys., Man, Cybern 20, 2, 404434, 1990.

[5] Wang, L. X., "Fuzzy systems are universal approximators", Proc. IEEE

Int. Conf. Fuzzy Systems, 11631170, 1992.

[6] N. Nariman-Zadeh, A. Darvizeh, M.H., "DadfarmaiDesign of ANFIS

networks using hybrid genetic and SVD methods for the modelling of

explosive cutting process", Journal of Materials Processing Technology

155–156, 1415 – 1421, 2004.

[7] Hoffmann, F. and Nelles, O. "Genetic Programming for Model Selection

of TSK-Fuzzy Systems", Information Sciences, 136, 728. Elsevier.

Germany, 2001.

[8] Wang, L. and Yen, J., "Extracting fuzzy rules for system modelling

using a hybrid of genetic algorithms and Kalman filter", Fuzzy Sets and

Systems, 101, 353-362, 1999.

[9] Jang, J. S. R., "ANFIS: Adaptive-network-based fuzzy inference

system", IEEE Trans. Sys., Man, Cyb., 23, 665685, 1993.

[10] N. Nariman-zadeh, F. Kalantary, A. Jamali, F. Ebrahimi, " Robust

Pareto Design of GMDH-type Neural Networks for Systems with

Probabilistic Uncertainties", International Conference on Inductive

Modelling ICIM’2008, September 15–19, 2008, Kyiv, Ukraine

[11] Papadrakakis, M., Lagaros, N.D., Plevris, V., Design optimization of

steel structures considering uncertainties, Engineering structure, vol. 27,

1408-1418, 2005

[12] Kalos, M.H., Whitlock, P.A., "Monte Carlo Methods", Wiley, New

York, 1986.

[13] Diwekar, U.M., Kalagnaman, J.R., "Efficient sampling technique for

optimization under uncertainty", American Institute of Chemical

Engineering Journal, Vol. 43, n.2, pp. 440-447, 1997.

[14] Coello Coello, C. A., and Christiansen, A. D., "Multiobjective

optimization of trusses using genetic algorithms. Computers &

Structures", 75, pp. 647-660, 2000.

[15] Coello Coello, C. A., Van Veldhuizen, D. A. and Lamont, G. B.,

“Evolutionary Algorithms for Solving Multi-objective problems”,

Kluwer Academic Publishers, NY, 2002.

[16] Jamali, A., Nariman-Zadeh, N., Atashkari, K., “Multi-objective Uniform

diversity Genetic Algorithm (MUGA)”, In Advanced in Evolutionary

Algorithms, witold kosinski (Ed.), IN-TECH, Vienna, 2008.

[17] Jamali, A., Hajiloo, A., and Nariman-zadeh, N., Reliability-based robust

Pareto design of linear state feedback controllers using a multi-objective

uniform-diversity genetic algorithm (MUGA), Expert systems with

Applications, Vol. 37 (1) , pp. 401-413, 2010.

[18] Smith, B.A., Kenny, S.P., Crespo, L.G., Probabilistic Parameter

Uncertainty Analysis of Single input Single Output Control Systems,

NASA report, TM-2005-213280, March, 2005.

[19] Nariman-zadeh N. And Jamali A., "Pareto genetic design of GMDH-

type neural networks for nonlinear systems", IWIM’07, Prague, Czech

Rep, 2007.

304

- 1-s2.0-S1569190X15001689-mainUploaded byKhairul
- Business OptimizationUploaded bySushant Vanmali
- Chapter 4Uploaded byAthir03
- Path Planning Optimization Using ADAMS and HEEDSUploaded byIsrar Ullah
- Bv Cvxbook Extra ExercisesUploaded byAnonymous WkbmWCa8M
- FSMVRPTWUploaded byDuvan Edu
- EPFL_TH2572Uploaded byNguyễn Hữu Phấn
- VI-03 Minimum Variance Method_2007Uploaded byAamir Habib
- hw_1Uploaded byPrafessar Khaos
- 177519539 Alex Burtzos Beethoven Op 106 Hammerklavier Analysis With ExamplesUploaded byJaume Eduardo Lan
- Brochure-integrated-container-terminal-planning-optimization-EN.pdfUploaded byHerum Manalu
- f 111035812719462Uploaded byHadiBies
- A study on Crash and Impact Strength Analysis of Structural component of the vehicle for Occupant SafetyUploaded byInternational Journal for Scientific Research and Development - IJSRD
- Kf 3418331844Uploaded byAnonymous 7VPPkWS8O
- Solving Economic Dispatch Problem with Valve-Point Effect using a Modified ABC AlgorithmUploaded byAamirNawaz
- Manual de MPL.pdfUploaded bypraxtorres52
- Blend ExpertUploaded byMohammed Alkhafaji
- Cluster Optimization Rule_Ver1Uploaded byDpkKSingh