Fuzzy Sets and Systems 63 (1994) 45-55 45

North-Holland

Fuzzy linear regression with fuzzy intervals

Georg Peters
RWTH Aachen, lnstitut fiir Wirtschafiswissenschaften, Lehrstuhl fiir Operations Research, D-52056 Aachen, Germany

Received October 1993

Revised December 1993

Abstract: In this paper, we introduce a new class of fuzzy linear regression models based on Tanaka's approach. Unlike in the
Tanaka model, here all training data influence the estimated interval. The (crisp) LP of Tanaka's approach is generalized to a
fuzzy linear programming problem. With the proposed model, an adaptation of the fuzzy regression equation to new data
becomes possible.

Keywords: Fuzzy regression analysis; fuzzy linear programming; averaging operators; adaptive fuzzy regression analysis.

1. Introduction

1.1. Statistical linear regression

For many years statistical linear regression has been used in almost every field of science, e.g.
business forecasting, economics, natural sciences, engineering, and so on. The purpose of regression
analysis is to explain the variation of a dependent variable y in terms of the variation of explanatory
variables x = (xl, x 2 , . . . , XN) as: y----f(x) where f(-) is a linear (crisp) function. The use of statistical
linear regression is bounded by some strict assumptions (statistical properties) about the given data; e.g.
the values of the unobserved error term are mutually independent and identically distributed [1]. It is
also assumed that the explanatory and dependent data, and the relationship f(.) between them are
crisp. As a result, the statistical regression model can be applied only if the given data are distributed
according to a statistical model, and the data and the relationship between x and y are crisp.

1.2. Fuzzy linear regression

In fuzzy linear regression that was introduced by Tanaka et al. in 1982 [7], some of the strict
assumptions of the statistical model are relaxed. A collection of recent papers dealing with several
approaches to fuzzy regression analysis can be found in [2]. In ~this paper we concentrate on the model
of Tanaka.
In the general fuzzy regression model the input data (explanatory data x) and the output data
(dependent data y) are fuzzy, the relationship between the input and output data is given by a fuzzy
function, and the distribution of the data is possibilistic. They need not have statistical properties [8]. So
the fuzzy regression analysis should be applied to many real life problems in which the strict
assumptions of classical (statistical) regression analysis cannot be met.

1.3. Organization of the paper

An extension to Tanaka's model is presented in this paper. In the new model, the estimated interval
is influenced by all training data. Therefore, the model is robust in the presence of outliers.
In Section 2, Tanaka's model is introduced and the interpretation of the model as interval estimation
is presented. In Section 3 a new model with fuzzy intervals is presented. A comparison between
Tanaka's and the new model is given in Section 4. The paper ends with a short conclusion in Section 5.

Correspondence to: G. Peters, RWTH Aachen, Institut fur Wirtschaftswissenschaften, Lehrstuhl fur Operations Research,
D-52056 Aachen, Germany.

0165-0114/94/$07.00 (~) 1994--Elsevier Science B.V. All rights reserved

SS D I 0165-0114(94)00026-4
46 G. Peters / Fuzzy linear regression with fuzzy intervals

2. Tanaka's model

2.1. Foundations of the model

The basic model assumes a linear fuzzy function [5]:
Y =AoxoAlx~ +" • • +Ajxj +. • • +ANXN = A x (Xo:-- 1), (1)
where x is the (crisp) vector of the independent variables, and A is the (fuzzy) vector of the model's
parameters.
The A t may be represented in the form of triangular fuzzy numbers:

Aj(aj)
=
{1 I~j-a~l -
cj
, o9-cj<~aj<~aj+c j,
(2)
0, otherwise,
where Aj(aj) is the membership function, % is the center, and cj the spread of the fuzzy number.
Applying the extension principle we get
lY -xtal
1 ff~ , x#0,
Y(Y)= 1, x = 0 , y~a0, (3)
0, x = 0 , y =0.
One may choose to minimize the total vagueness of the given training data by

j=O i=l

(where M is the number of training samples) such that the membership degree of each observation is
greater than an imposed threshold:
Y(y~)>ih (i = 1, 2 , . . . , M). (5)
The above leads to the following linear programming problem, where L(.) represents the chosen form
of the fuzzy number A(.). ~

MIN ~ (cj ~ lx~j[)

j=o i=1
such that (6)
N N
~jxij + IL-l(h)l ~ cj Ixijl >~Yi,
j=O j=O
N N
Z OljXij--Ig-'(h)l ~ cj Ixijl ~<Y,
j=O j=O

C ~ O, Ol E ~) 1,
Xio : =
(O~h ~ 1;i = 1, 2, . . . , M).

2.2. Fuzzy regression analysis as interval estimation

Tanaka's model can be interpreted as interval estimation of the dependent variable y [6]. At first an
interval is calculated that just covers all training data y. In the second step a membership function may
be constructed with respect to the interval. For example we assume that the shape of the membership
function is triangular and that the membership degrees in the interval are higher than 0.3 as in

J In the case of the triangular fuzzy n u m b e r we h a v e IL(h) ~l = 1 - h ; (h e [0, 1]).

 G. Peters / Fuzzy linear regression with fuzzy intervals 47

A:
membership
Y P function:

I a
b
0.65

0.3

Fig. 1. Interval estimation and membership functions.

membership function a of Figure 1. In Figure 1 the membership function b represents an alternative

selection by a user.
It is assumed in Tanaka's model that all training data y are in the interval. In other words, the bounds
of the interval are determined by the 'worst' data of a given training set. This assumption can be
justified if there are only a few training samples because one may not be able to decide whether there

(a) (b)

~'-
v
Ib,
X ×
Fig. 2. Crisp intervals and outliers.

is an outlier in the set or not (e.g., Figure 2(a)). On the other hand, this assumption leads to bad
estimations if we have 'enough' training data with just one (or a few) outlier(s) (e.g., Figure 2(b)).
We can differentiate between two kinds of intervals (Figure 3). The first interval just covers the

Y Y

I • support
/
(h =0)

y
/ v

X X
Fig. 3. Interval and support.
48 G. Peters / Fuzzy linear regression with fuzzy intervals

training data, and the second is defined by the h-cut, h = 0. The latter equals the support of Y. The
support depends on the imposed threshold h and the shape of the fuzzy number A(.). Different
thresholds h and different shapes of A(.) lead to different supports. The width of the support is
interpreted as a confidence factor in the training data by the user of the model. A wide support means
that the user is pessimistic about the training data covering the universe of the real data [9].
Thus, the first step of fuzzy regression is interpreted as interval estimation. In the second step a
membership function may be constructed, which determines the support (see Figure 1).
It is apparent that this interpretation coincides with a theorem given by Moskowitz and Kim [3]. If
one regression analysis solution, Ah,,L
* , = (a*, c*), is known then other solutions are obtained by (see
Figure 1):
IZll(hl)l
A*2,L2= (a*, c ). (7)

In the following sections we concentrate on the first step of fuzzy regression analysis. We estimate an
interval that just covers the training data, that makes IL-l(h)r := 1. Until now the interval is crisp, the
bounds are determined by just a few (bad) values. A compensation between good and bad training data
is impossible. Therefore, Tanaka's model is very sensitive to outliers.

3. A model with fuzzy intervals

3.1. Fuzzy intervals

In the new model, we assume that the bounds of the interval are fuzzy. The dependent data y are no
longer inside or outside the interval but belong to the interval to certain degrees (memberships).
Therefore, compensations between 'good' and 'bad' data become possible. Outliers are compensated by
data that lie in the interval, and the estimated interval is determined by all d a t a - not only by the
'worst'.
There is a trade-off between the objective function (MIN X spread) and the boundary of the interval.
A very wide interval includes almost all real data (which is good), but we get a high value of the
objective function (which is bad) and vice versa. Since we have a symmetric problem, we formulate the
objective function also as fuzzy. We obtain a fuzzy linear programming problem:

MIN (c/ ~ lxi/) (8)

1"=0 i=l
such that
N N
2
.i=O
o~jx~j+ ~ c: Ix;El~yi,
j=o
N N
oqxi: - ~] cj Ix~jl~<y,
j=O j=O
c/>0, o~O~, x~0:=l,
(IZ-l(h)l :-- 1;i = 1, 2 , . . . , M).
To review fuzzy linear programming, one must express the problem via the following mathematical
model [10]:
~-KX ctx ctx ~ z
Ax <~b ~ Ax <. b (9)
x >~O x >~O,
where z represents a desired lower bound of the objective function.
 G. Peters / Fuzzy linear regression with fuzzy intervals 49

/
1

v
v
z-p o z b b+p
Fig. 4. Fuzzy constraints.

Introducing a new variable A which represents the membership degree to which the solution belongs
to the set 'good solution', it is easy to show that the above formulation can be written as (see Figure 4)

MAX A
s.t. Apo -- ctx ~ --Z,

Api + (Ax)i <~bi + Pi,

(lo)
O<~A<~I,
x I>0,
(i = 1, 2 . . . . , M, M is the number of constraints),

where p~ is the width of the 'tolerance interval'. Applying the above fuzzy linear programming model to
(8) we get

MAX a (11)
such that
M N
(1 - A)po - ~ ~ cj [xol >I -do ('objective function'),
i=l j=O
N N
(1 - A)pi + ~ ajxij + ~ cj Ixijl ~Yi (upper limit),
j =0 j=o
N N
(1 -- A)p i -- E OtjXij "4- E Cj Ix,jl I> -y, (lower limit),
j~o j=o

- A ~> - 1 ,
Z,c~>O, a ~ R , X~o:=l,
([L-'(h)] := 1;i = 1, 2 . . . . , M),

where Pi is the width of the tolerance interval of datum yi. The parameters P0 and Pi must be
determined in a context dependent way (e.g. see Section 4, Example 1). The parameter do represents
the desired value of the objective function (represented by A = 1). We suggest to select do = 0 which
makes ~ = o (cj ~ [xvI)= 0 for the desired value of the total vagueness; thus we prefer to obtain a
model as crisp as possible.
50 G. Peters [ Fuzzy linear regression with fuzzy intervals

In (11))t is determined by a trade-off between the objective function (minimization of the spread)
and the equation of the 'worst' datum y. This equation requires the lowest values of ~ in comparison to
the other equations. Therefore, we use MIN as the aggregation operator. Note that there is still no
compensation between 'good' and 'bad' data.

3.2. Selection of an aggregation operator

We may modify the fuzzy linear program and use the arithmetic mean [10] as the aggregation
operator. This leads to
1
MAX ~ = M i=1 /~i (12)

such that
M N
(1 - X)po - ~ ~ cj Ix,jP>~-do,
i=1 j~0
N N
(1 - Ai)pi + ~_~ otixij + ~ cj Ixijl ~ yi,
j~O j=O
N N
(1 - Ai)p - jxij + cj Ixijl
/=o j=o
--h i/> -1,
t~, C ~- O, Ol ~ ff~, Xio := 1,
([L-l(h)l := 1;i -- 1, 2 , . . . , M).
Now a compensation between good and bad training data is achieved. Each training datum influences
the regression function with a weight of 1/M.

3.3. Selection of the range of A

In the fuzzy linear program (12) we have fuzzy intervals but still crisp overall bounds because the Ai
are restricted to [0, 1]. The restriction of A to [0, 1] makes sense if one defines utilities. We define the
maximal utility by ~ = 1 and no utility by A = 0 as in the following example. Suppose we are fully
satisfied if the profit of a project is $1.5 mio. or higher (A --- 1). But it must be higher than $1.0 mio.
otherwise we are totally discontented (~ = 0) (see Figure 5). However, in our case the restriction
e [0, 1] does not make sense because we measure distances and not utilities. Therefore we are free to
select the range of A in several ways.
For example: (i) Data that lie inside the interval have membership degrees of ~ = 1. Data outside the
interval have membership degrees less than 1. There is no lower bound for A. Data that are

v
1.0 1.5 $/mio.
Fig. 5. Utility of the profit.
 G. Peters / Fuzzy linear regression with fuzzy intervals 51

0) (ii)

/
Fig. 6. Some membership functions.

infinitely far away from the boundary of the interval have membership degrees of A-* -0o (see Figure
60)).
A can be interpreted easily. We assume a fuzzy system. The system contains vagueness to a certain
degree. Therefore we do not distinguish training data that are 'quite good' in the sense that they are in
the interval. They get the same highest value of A (A = 1). We assume that the worst data of a training
s e t - data that are not in the interval- cannot be explained totally by the vagueness of the system.
Therefore these data have lower membership degrees (A < 1).
To obtain normalized membership degrees one may use the mapping A e ( - ~ , 1 ] ~ A . . . . . lized @
[0, 1]. For example: A. . . . lized = 1/(2 -- A).
(ii) Data which are located on the mean a get the highest membership degrees (even higher than 1).
Again there is no lower bound for A: A ~ ( - ~ , + ~ ) (see Figure 6(ii)). This fuzzy regression model is
related to the statistical Ll-criterion.

3.4. Generalizations of the model

There are several ways to generalize the model. Some possibilities are listed below.
(i) Until now we assumed symmetric properties with respect to the upper and lower bound of the
interval. But one can split Ai into A~upperb°und) and All. . . . bound). In the same way we can generalize Pi
and c,.
(ii) The solution of the LP provides us with the values of the A~:
- T h e values of the /~i may help in constructing membership functions L(-) that cover the
interval.
- A very low value of a special )% means that the y~,, is far outside the interval.
- A special )t~omight be much smaller than A.
In the latter two cases one can easily detect (x, Y)io as an outlier and deal with it - for example one can
remove the outlier and solve the linear program again.
(iii) We assumed that each training datum influences the regression function with a weight of 1/M
(A = ( l / M ) ~]~1 Ai). We can easily construct an adaptive fuzzy regression model by allowing the new
data to have a higher influence on the objective function than the old data. This can be achieved by
formulating a new A:
M M
,~ = ~ w,A~, ~ w~ = 1, (13)
i-1 i=1
52 G. Peters / Fuzzy linear regression with fuzzy intervals
Table 1. Data of ex-
periment 1

x y x y

1 1.5 6 6.3
2 2.3 7 6.5
3 2.7 8 7.8
4 4.4 9 8.5
5 9.4 10 10.5

where wi is a weighting parameter, which is low for old data and high for new data.

4. Applications of the model

As example, we present a model with fuzzy intervals which is formulated in the following way
(Section 3.3, case (i)): 2

1 M
MAX X : ~ E/~i (14)
i-----1

such that
M N
(1 - '~)Po - ~ ~ cj Ix~jl>I -do,
i=1 j=O

N N
(I - ~i)Pi -'}-~ oljxij -]- ~ cj IxoI ~ Yi,
j=o 1=o

N N
(1 - Ai)pl - ~ ajxij + ~ cj Ixol >t -Yi,
j=o j=o

c~>0, a e R , A~<I, Xio:=l,

(IL-l(h)l := 1;i = 1, 2 , . . . , M).

Two experiments are presented. In the first experiment we use a training set of 10 samples. In the
second experiment we present a real-life application with OECD-data.

Table 2..Parameters of experiment 1

do Po Pl =P2 =P3 . . . .

0 1000 1
0 100 10

2 The other formulations can be constructed and interpreted easily.

 G. Peters / Fuzzy linear regression with fuzzy intervals 53

y--f (x)
16

j J
J
i i i i

Fig. 7. Tanaka's model.

4.1. Experiment 1

We assume the following linear regression function: y = Ao + A I " X . The training data are given in
Tables 1 and 2. The results are shown in Figures 7-9. There is one outlier in the training data, where
x = 5. In Tanaka's model the interval covers all training data, including the outlier. Therefore the
estimated interval is 'too' wide. We obtain a bad estimation for the given data set (see Figure 7).
The results of the new model are shown in Figures 8, 9. The outlier is compensated by the other
training data. The width of the estimated interval depends on the selection of the parameters do, Po, Pi.
y=f(x)
16
14
12
10
8
j J J .~
J
J J
....:~J
J

• J i

Fig. 8. d o = 0 , po = 1000, P l = P 2 . . . . . p l o = 1.

A weak requirement to minimize the spread (= a high value of Po and low values of Pi) leads to a wide
interval (see Figure 8). Strong requirements to minimize the spread (= a low value of Po and high
values of p;) lead to a narrow interval (see Figure 9).

4.2. Experiment 2

We choose do = 0, Po = l e 5 , Pl . . . . . P24 = 1. We use OECD-data of the U S A [4] as given in Table

3. The fuzzy linear function is given by cpt+6 = Ao + AI • cpt + A2 " ipt + m3 " net.
The results of the fuzzy linear regression are enumerated in Table 4. The estimated interval is

y=f(x)
12

10

-2
1 2 3 4 5 6 7 8 9 10
X

Fig. 9. d o = 0 , po = 100, P l = P 2 . . . . . p~o = 10.

54 G. Peters / Fuzzy linear regression with fuzzy intervals

Table 3. O E C D data

t (date) Consumer Industrial New t (date) Consumer Industrial New

prices production consumer prices production consumer
(cp) (ip) credits (nc) (cp) (ip) credits (nc)

1 (01.1%0) 54.5 57.6 3.63 16 (04.1961) 55.4 55.3 4.00

2 (02.1960) 54.6 57.9 3.81 17 (05.1961) 55.4 56.1 4.44
3 (03.1960) 54.4 57.7 4.10 18 (06.1961) 55.5 57.7 4.60
4 (04.1960) 54.9 57.3 4.49 19 (07.1961) 55.7 54.8 4.18
5 (05.1960) 54.9 57.1 4.40 20 (08.1961) 55.6 57.6 4.53
6 (06.1960) 55.0 57.3 4.60 21 (09.1961) 55.8 59.0 3.%
7 (07.1960) 55.1 53.8 4.23 22 (10.1961) 55.8 60.4 4.48
8 (08.1960) 55.1 55.7 4.49 23 (11.1961) 55.8 59.9 4.48
9 (09.1%0) 55.1 56.5 4.12 24 (12.1961) 55.7 58.9 5.06
10 (10.1960) 55.3 56.8 4.10 25 (01.1962) 55.7
11 (11.1960) 55.4 54.8 4.16 26 (02.1962) 55.9
12 (12.1960) 55.4 52.4 4.71 27 (03.1962) 56.0
13 (01.1961) 55.4 52.6 3.49 28 (04.1962) 56.1
14 (02.1%2) 55.4 53.2 3.31 29 (05.1962) 56.1
15 (03.1962) 55.4 53.9 4.09 30 (06.1962) 56.2

Table 4. Results of experiment 2

ao = 14.013 hi = 0.99999 A9 = 1.0 /~17 = 1.0

al = 0.732 A2 = 1.0 hlo = 1.0 A18 = 1.0
a2 = 0.009 A3 = 1.0 At1 = 0.99880 A 1 9 = 1.0
a 3 = 0.139 h4 = 1.0 A12 = 0.99999 )~20 = 1.0
CO= 0.0 AS = 0.99999 A13 = 1.0 A21 = 0.99999
cl = 0.004 A6 = 0.99999 hla = 0.99999 A22 = 0.99999
c2 = 0.0 h 7 = 1.0 hi5 = 1.0 /~'23 = 1.0
C3 = 0 . 0 /~8 = 1.0 h i 6 = 1.0 '~24 = 1.0

influenced by all training data. All training data are either in the estimated interval (2, = 1) or very close
to it (A = 0.99. • .). Therefore we can conclude that there is no outlier in the data.

5. Conclusion

A fuzzy regression model with fuzzy intervals has been introduced. It has been shown that all training
data influence the estimated interval. It has been demonstrated that this model is a useful generalization
of the model of Tanaka.
The extended model should be used in interaction with an expert. The various parameters that can be
chosen and the several ways to formulate the regression analysis make it possible for the expert to add
knowledge to the model that is not contained in the data itself. Therefore it is a very flexible instrument
which can be applied to many real-life applications.

References

[1] J.D. Jobson, Applied Multivariable Data Analysis, Vol. 1 (Springer Verlag, New York, 1991).
[2] J. Kacprzyk and M. Fedrizzi, Fuzzy Regression Analysis (Physica-Verlag, Heidelberg, 1992).
[3] H. Moskowitz and K. Kim, On assessing the h value in fuzzy linear regression, Fuzzy Sets and Systems 58 (1993) 303-327.
[4] OECD, Main economic indicators 1960-1979 (OECD, Paris, 1980).
[5] W. Pedrycz and D.A. Savic, Evaluation of fuzzy regression models, Fuzzy Sets and Systems 39 (1991) 51-63.
 G. Peters / Fuzzy linear regression with fuzzy intervals 55

[6] H. Tanaka and H. Ishibuchi, Possibilistic regression analysis based on linear programming, in: J. Kacprzyk and M. Fedrizzi
(Eds.), Fuzzy Regression Analysis (Physica-Verlag, Heidelberg, 1992) 47-80.
[7] H. Tanaka, S. Uejima and K. Asai, Fuzzy linear regression models, IEEE Trans. Systems, Man and Cybernet. 12 (1982)
903-907.
[8] H. Tanaka, Fuzzy data analysis by possibilistic linear models, Fuzzy Sets and Systems 24 (1987) 363-375.
[9] H. Tanaka and J. Watada, Possibilistic linear systems and their applications to the linear regression model, Fuzzy Sets and
Systems 27 (1988) 275-289.
10] H.-J. Zimmermann, Fuzzy Sets, Decision Making, and Expert Systems (Kluwer Academic, Boston, 1987).