Approximation of fuzzy neural networks

by
using Lusin’s theorem

Jun Li∗ Jianzeng Li Masami Yasuda

School of Sciences, School of Information Engi., Faculty of Science,
Commun. Univ. of China, Commun. Univ. of China, Chiba University,
Beijing 100024, China Beijing 100024, China Chiba 263-8522, Japan
lijun@cuc.edu.cn jzli@cuc.edu.cn yasuda@math.s.chiba-u.ac.jp

Abstract is, any fuzzy-valued measurable function can

In this note, we study an approxima-
tion property of regular fuzzy neu-
ral network(RFNN). It is shown that
any fuzzy-valued measurable func-
tion can be approximated by the
four-layer RFNN in the sense of
fuzzy integral norm for the finite
sub-additive fuzzy measure on R.
Keywords: Fuzzy measure; Lusin’s
theorem; Approximation; Regular
fuzzy neural network
1 Introduction
In neural network theory, the learning ability
of a neural network is closely related to its
approximating capabilities, so it is important
and interesting to study the approximation
properties of neural networks. The studies on
this matter were undertaken by many authors
and a great number of important results were
obtained ([1, 4, 13] etc). The similar approx-
imation problems in fuzzy environment were
investigated by Buckley [2, 3], P. Liu [7, 8] and
other authors. In [8] Liu proved that contin-
uous fuzzy-valued function can be closely ap-
proximated by a class of regular fuzzy neu-
ral networks (RFNNs) with real input and
fuzzy-valued output. In this note, by using
Lusin’s theorem on fuzzy measure space, we
show that such RFNNs is pan-approximator
for fuzzy-valued measurable function. That
∗
Supported by the postdoctoral research finance of
Communication University of China.
be approximated by the four-layer RFNNs in
the sense of fuzzy integral norm for the finite
sub-additive measure on R.
2 Preliminaries
We suppose that (X, ρ) is a metric space, and
that O and C are the classes of all open and
closed sets in (X, ρ), respectively, and B is
Borel σ-algebra on X, i.e., it is the smallest
σ-algebra containing O.
A set function µ : B → [0, +∞] is called a
fuzzy measure([11]), if it satisfies the following
properties: (FM1) µ(∅) = 0; (FM2) A ⊂ B
implies µ(A) ≤ µ(B). A fuzzy measure µ is
called null-additive ([15]), if for any E, F ∈ B
and µ(F ) = 0 imply µ(E ∪ F ) = µ(E); sub-
additive ([12]), if for any E, F ∈ B we have
µ(E ∪ F ) ≤ µ(E) + µ(F ).
In this paper, we always assume that µ is a
finite, sub-additive and continuous fuzzy mea-
sure on B.
Consider a nonnegative real-valued measur-
able function f on A and the fuzzy integral of
f on A with respect to µ, which is denoted by
Z
(S) f dµ
A
_
, [α ∧ µ({x : f (x) ≥ α} ∩ A)]
0≤α<+∞
Theorem 2.1 (Lusin’s theorem cf. [6, 14])
Let (X, ρ) be metric space and µ be null addi-
tive fuzzy measure on B. If f is a real-valued
measurable function on E ∈ B, then, for every
² > 0, there exists a closed subset F² ∈ B such (1) d(Ã · Ã1 , Ã · Ã2 ) ≤ |Ã| · d(Ã1 , Ã2 ),
that f is continuous on F² and µ(E − F² ) < ². n n n
X X X
(2) d( W̃i , Ṽi ) ≤ d(W̃i , Ṽi ).
3 Approximation in fuzzy mean by i=1 i=1 i=1
regular fuzzy neural networks
By the well known extension principle, each
In this section, we study an approximation function f : Rn → R may be extended to one
property of the four-layer RFNNs to fuzzy- F0 (R)n → F(R) and, for each fuzzy number,
valued measurable function in the sense of the addition, the multiplication and the mul-
fuzzy integral norm for fuzzy measure on R. tiplication by a scalar are defined by the ex-
tension princile ([8]).
Let F0 (R) be the set of all bounded fuzzy
numbers, i.e., for à ∈ F0 (R), the following Let T be a measurable set in Rn , (T, B ∩ T, µ)
conditions hold: finite fuzzy measure space. Let L(T ) denote
the set of all fuzzy-valued measurable function
(i) ∀ α ∈ (0, 1], Ãα , {x ∈ R | Ã(x) ≥ α} is F̃ : T → F0 (R).
the closed interval of R;

(ii) The support Supp(Ã) , cl{x ∈ R |
Ã(x) > 0} ⊂ is a bounded set;
(iii) {x ∈ R | Ã(x) = 1} 6= ∅.
For simplicity, supp(Ã) is also written as Ã0 .
For any F˜1 , F˜2 ∈ L(T ), d(F˜1 , F˜2 ) is measur-
able function on (T, B ∩ T ), we will write a
fuzzy integral norm as
Z
4S (F˜1 , F˜2 ) , (S) d(F˜1 , F˜2 )dµ.
T

Obviously, Ã0 is a bounded and closed inter- Proposition 3.2 Let F˜1 , F˜2 , F˜3 ∈ L(T ),
val of R. For à ∈ F0 (R), let Ãα = [a− +
α , aα ] then
for each α ∈ [0, 1] and we denote
_ 4S (F˜1 , F˜3 ) ≤ 2(4S (F˜1 , F˜2 ) + 4S (F˜2 , F˜3 )).
|Ã| , (|a− +
α | ∨ |aα |).
α∈[0,1] Proof. From subadditivity of µ, we have
Z
For Ã, B̃ ∈ F0 (R), define metric d(Ã, B̃) be- 4S (F1 , F3 ) = (S) d(F˜1 , F˜3 )dµ
˜ ˜
T
tween à and B̃ by
_
_ = {α ∧ µ(T ∩ (d(F˜1 , F˜3 ))α }
d(Ã, B̃) , dH (Ãα , B̃α )
α∈[0,∞)
α∈[0,1]
_
≤ {α ∧ µ(T ∩
where dH means Hausdorff metric: for A, B ⊂
α∈[0,∞)
R,

dH (A, B) (d(F˜1 , F˜2 ) α2 ∪ d(F˜2 , F˜3 ) α2 ))}

( )
, max sup inf (|x − y|), sup inf (|x − y|) .
x∈A y∈B y∈B x∈A
It is known that (F0 (R), d) is a completely
separable metric space ([5]). Also we note
that the next assertion which is used in later.
_
≤ {α ∧ [µ(T ∩ d(F˜1 , F˜2 ) α2 )
α∈[0,∞)
+µ(T ∩ d(F˜2 , F˜3 ) α2 )]}.
Because of the elementary inequality: a ∧ (b +
c) ≤ (a ∧ b) + (a ∧ c) where a, b, c ≥ 0, we have

4S (F˜1 , F˜3 )
Proposition 3.1 ([8]) Assume Ã, Ã1 , Ã2 ∈ _
F0 (R), and W̃i , Ṽi ∈ F0 (R)(i = 1, 2, · · · , n). ≤ {α ∧ µ(T ∩ d(F˜1 , F˜2 ) α2 )
Then α∈[0,∞)
 +α ∧ µ(T ∩ d(F˜2 , F˜3 ) α2 )} Proposition 3.3 Let µ be a finite, sub-
_ additive and continuous fuzzy measure on R.
≤ [α ∧ µ(T ∩ d(F˜1 , F˜2 ) α2 )] If F̃ ∈ L(T ), then for every ² > 0, there exists
α∈[0,∞) Φ̃² ∈ S(T ) such that
_
+ [α ∧ µ(T ∩ d(F˜2 , F˜3 ) α2 )] 4S (F̃ , Φ̃² ) < ².
α∈[0,∞)
_ hα Define
≤ ∧ µ(T ∩ d(F˜1 , F˜2 ) α2 ) ( ¯ )
2 ¯ n
α∈[0,∞) ¯ X
i H[σ] , H̃ ¯ H̃(x) = W̃i Vi [σ]
α ¯
+ ∧ µ(T ∩ d(F˜1 , F˜2 ) α2 ) i=1
2
_ hα where
+ ∧ µ(T ∩ d(F˜2 , F˜3 ) α2 )
2 m
X
α∈[0,∞)
Vi [σ] , Ṽij · σ(x · Ũj + Θ̃j )
α i j=1
+ ∧ µ(T ∩ d(F˜2 , F˜3 ) α2 )
2
_ hα i and σ, by the same notation, is a given ex-
≤ ˜ ˜
∧ µ(T ∩ d(F1 , F2 ) 2 )α tended function of σ : R → R (bounded,
α
2 continuous and nonconstant), and x ∈
2
∈[0,∞)

_ hα i R, W̃i , Ṽij , Ũj , Θ̃j ∈ F0 (R).

+ ∧ µ(T ∩ d(F˜1 , F˜2 ) α2 ) For any H̃ ∈ H[σ], H̃ is a four-layer feed-
α
2
∈[0,∞)
2 forward RFNN with activation function σ,
_ hα i threshold vector (Θ̃1 , . . . , Θ̃m ) in the first hid-
+ ∧ µ(T ∩ d(F˜2 , F˜3 ) α2 )
α
2 den layer(cf. [8]).
2
∈[0,∞)

_ hα i Restricting fuzzy numbers Ṽij , Ũj , Θ̃j ∈

+ ∧ µ(T ∩ d(F˜2 , F˜3 ) α2 ) F0 (R), respectively, to be real numbers
α
2 vij , uj , θj ∈ R, we obtain the subset H0 [σ] of
2
∈[0,∞)
³ ´ H[σ]:
= 2 4S (F˜1 , F˜2 ) + 4S (F˜2 , F˜3 ) . ( ¯ )
¯ Xn
¯
H0 [σ] , H̃ ¯ H̃(x) = W̃i vi [σ] .
Definition 3.1 ([8]) A fuzzy-valued func- ¯
i=1
tion Φ̃ : T → F0 (R) is called a
fuzzy-valued simple function, if there exist where
Ã1 , Ã2 , . . . , Ãm ∈ F0 (R), such that ∀ x ∈ T , m
X
vi [σ] , vij · σ(x · uj + θj ).
m
X j=1
Φ̃(x) = Ãk · χTk (x)
k=1
Let define two classes of pan-approximation
where Tk ∈ B ∩ T (k which is fundamental to our results.
S = 1, 2, . . . , m), Ti ∩ Tj =
∅ (i 6= j) and T = mk=1 Tk .
Definition 3.2 (1) H0 [σ] is call the pan-
approximator of S(T ) in the sense of 4S ,
Immediately, if S(T ) denotes the set of all
if for ∀ Φ̃ ∈ S(T ), ∀ ² > 0, there exists
fuzzy-valued simple functions, then S(T ) ⊂
H̃² ∈ H0 [σ] such that 4S (Φ̃, H̃² ) < ².
L(T ).
Similar to the proof of Proposition 3.2 and by (2) For F̃ ∈ L(T ), H[σ] is call the pan-
using subadditivity of µ, we can obtain the approximator for F̃ in the sense of 4S ,
following proposition. if ∀ ² > 0, there exists H̃² ∈ H[σ] such
that 4S (F̃ , H̃² ) < ².
By using Lusin’s theorem (Theorem 2.1),
Proposition 3.2 and 3.3 we can obtain the ( 0
vk(j−βk ) , if βk < j ≤ βk+1 ,
main result in this paper, which is stated in vkj =
the following. 0 otherwise,
( 0
Theorem 3.1 Let (T, B ∩ T, µ) be fuzzy mea- θk(j−βk ) , if βk < j ≤ βk+1 ,
θkj =
sure space and µ be finite, sub-additive and 0 otherwise,
continuous. Then, ( 0
wk(j−βk ) , if βk < j ≤ βk+1 ,
(1) H0 [σ] is the pan-approximator of S(T ) in wkj =
0 otherwise,
the sense of 4S .
then, for any k ∈ {1, 2, · · · , m}, we have
(2) H[σ] is the pan-approximator for F̃ in the Pp
sense of 4S . j=1 vij · σ(hwkj , xi + θkj )
Pk 0
= pj=1
0 0
vij · σ(hwkj , xi + θkj ).
Proof. By using the conclusion of (1) and
Proposition 3.3 we can obtain (2). Now we
Now denote that
only prove (1). Suppose that Φ̃(x) is a fuzzy-  
valued simple function, i.e., m
X Xp

m
H̃(x) = Ãk · vkj · σ(hwkj , xi + θkj ) ,
X k=1 j=1
Φ̃(x) = χTk (x) · Ãk (x ∈ T ).
k=1
then H̃ ∈ H0 [σ].
For arbitrarily given ² > 0, applying Theorem In the reminder part of this section we will
2.1 (Lusin’s theorem) to each real measurable prove 4S (H̃, Φ̃) < ². Denote
function χTk (x), for every fixed k (1 ≤ k ≤
m), there exists closed set Fk ∈ B ∩ T such Bkj = vkj · σ(hwkj , xi + θkj )
that
and
² 0 0 0 0
Fk ⊂ Lk and µ(Lk − Fk ) < Bkj = vij · σ(hwkj , xi + θkj ).
2m
By using Proposition 3.1 and noting µ(T −
and χTk (x) is continuous on Fk .
L) < ²/2, we have
Therefore, for every k there exist a
Tauber-Wiener function σ and pk ∈ 4S (H̃, Φ̃)
0 0 0 0 0 0
N, vk1 , vk2 , · · · , vkpk , θk1 , θk2 , · · · , θkpk ∈ R, Z
0 0 0
and wk1 , wk2 , · · · , wkpk ∈ Rn such that = (S) d(H̃, Φ̃)dµ
T
¯ pk ¯ _ h i
¯ X 0 0 0 ¯ ²
¯ χT (x)− v ·σ(hw , xi+θ ) ¯< = α ∧ µ(T ∩ (d(H̃, Φ̃))α ) .
¯ k kj kj kj ¯ m
X
j=1 0≤α<+∞
2 |Ãk |
k=1 Since
for
Pm x ∈ Lk . Note that we can assume µ(T ∩ d(H̃, Φ̃)α )
k=1 |Ãk | = 6 T0, without any loss of generality.  
Denote L = m k=1 Lk , then T = L ∪ (T − L). Xm p
X
By the subadditivity of µ, we have = µ T ∩ d  Ãk · Bkj ,
S k=1 j=1
µ(T − L) = µ( m k=1 (T − Lk )) ! !
m
X
P ²
≤ m k=1 µ(T − Lk ) < .
χTk (x) · Ãk
2 k=1 α
Pk−1 ¡ ¢
We take βP 1 = 0, βk = i=1 pi , k = 2, · · · , m, ≤ µ (L ∪ (T − L)) ∩ (Cmp )α
m
and p = p
k=1 k . For k = 1, 2, · · · , m, j =
1, 2, · · · , p, we denote ≤ µ (L ∩ (Cmp )α ) + µ ((T − L) ∩ (Cmp )α )
where the notation of set Cmp is assigned as Thus, combining with the previous evalua-
m
à p ! tion, we obtain
X X
Cmp = |Ãk | · d Bkj , χTk (x) . 4S (H̃, Φ̃) h
k=1 k=1 W ³ ¡ 0 ¢ ´i ²
≤ 0≤α<+∞ α ∧ µ L ∩ Dmp α +
Hence 4S (H̃, Φ̃) is dominated by 2
< ².
_ £ ¡ ¢¤
α ∧ µ L ∩ (Dmp )α
The proof of (1) is now completed. 2
0≤α<+∞
_
+ [α ∧ µ ((T − L))] References
0≤α<+∞
_ h ³ ¡ 0 ¢ ´i ² [1] J.G Attali, G. Pages, Approximation of
≤ α ∧ µ L ∩ Dmp α
+ functions by a multilayer perceptron: a
2
0≤α<+∞ new approach, Neural Networks 10(1997)
Here, for simplicity, we use the notations of 1069-1081.
¯ ¯
m ¯ p ¯ [2] J.J Buckley, Y. Hayashi, Fuzzy input-
X ¯X ¯
Dmp = ¯
|Ãk | · ¯ Bkj − χTk (x)¯¯ output controllers are universal approxi-
k=1 ¯ j=1 ¯ mators, Fuzzy Sets and Systems 58(1993)
and 273-278.
 
m
X pk
X 0
[3] J.J Buckley, Y. Hayashi, Can fuzzy
0
Dmp = |Ãk | · d  Bkj − χTk (x) . neural nets approximate continuous
k=1 j=1 fuzzy function, Fuzzy Sets and Systems
61(1994) 43-51.
Now we estimate the first part in the above
formula. If x ∈ L, then for every k = [4] R.M Burton, H.G. Dehling, Universal
1, 2, · · · , m, we have x ∈ Lk , hence approximation in p-mean by neural net-
¯ pk ¯ works, Neural Networks 11(1998) 661-
¯ X ¯ ²
¯ χT (x)− 0 0 0
vkj ·σ(hwkj , xi+θkj ) ¯¯< m , 667.
¯ k X
j=1 2 |Ãk | [5] P Diamond, P. Kloeden, Metric Spaces
k=1 of Fuzzy Sets, World Scientific Publish-
for every k = 1, 2, · · · , m. That is, for x ∈ L, ing, 1994.
 
Xm Xpk
0 ² [6] J. Li, M. Yasuda, Lusin’s theorem on
0
Dmp = |Ãk | · d  Bkj − χTk (x) < . fuzzy measure spaces, Fuzzy Sets and
2
k=1 j=1 Systems 146(2004) 121-133.
Therefore, [7] P. Liu, Analyses of regular fuzzy neural
_ h ³ ¡ 0 ¢ ´i
α ∧ µ L ∩ Dmp networks for approximation capabilities,
α Fuzzy Sets and Systems 114(2000) 329-
0≤α<+∞
338.
_ h ³ ¡ 0 ¢ ´i
= α ∧ µ L ∩ Dmp α [8] P. Liu, Universal approximations of con-
α∈[0, 2² ]
tinuous fuzzy-valued functions by multi-
_ h ³ ¡ 0 ¢ ´i layer regular fuzzy neural networks,
+ α ∧ µ L ∩ Dmp α Fuzzy Sets and Systems 119(2001) 313-
α∈[ 2² ,∞)
320.
_ h ³ ¡ 0 ¢ ´i
= α ∧ µ L ∩ Dmp α [9] P. Liu, Theory of fuzzy neural net-
α∈[0, 2² ] works and its applications, Ph.D. Thesis
² (in Chinese), Beijing Normal University,
≤ . 2002.
2
[10] P. Liu, Approximation analyses of feed-
forward regular fuzzy neural network
with two hidden layers, Fuzzy Sets and
Systems 150(2005) 373-396.

[11] Y. Narukawa, T. Murofushi, Regular

non-additive measure and Choquet inte-
gral, Fuzzy Sets and Systems 143(2004)
373−379.

[12] E. Pap, Null-additive Set Functions,

Kluwer, Dordrecht, 1995.

[13] F Scarselli, A.G. Tsoi, Universal approx-

imation using feedforward neural net-
works: a survey of some existing meth-
ods, and some new results, Neural Net-
works 11(1998) 15-17.

[14] J. Song, J. Li, Regularity of null-additive

fuzzy measure on metric spaces, Inter-
national Journal General Systems 2003,
Vol.33(3): 271-279..

[15] Z. Wang, G. J. Klir, Fuzzy Measure The-

ory, Plenum, New York, 1992.