Acai05 Berthold FL PDF

ALTANA Chair for Overview
Bioinformatics and Information Mining

• Fuzzy Logic
– Motivation
– Degrees of Membership and Fuzzy Sets
– Linguistic Values and Variables
Tutorial: Fuzzy Logic – Operators on Fuzzy Sets
– Fuzzy Implication
Advanced Course on Knowledge Discovery, Ljubljana, June 2005 • Fuzzy Rules
– Inference
– Construction of Fuzzy Rules from Data
Michael R. Berthold
Konstanz University • Fuzzy Arithmetic
– Fuzzy Numbers
Michael.Berthold@uni-konstanz.de – Operations on Fuzzy Numbers
– Extension Principle
• Conclusion
22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005. #2
Fuzzy Logic: Motivation Characteristic Functions: Crisp Sets

• Modeling of imprecise concepts: • Classical Sets can be described by a characteristic function:
– Age, Weight, Height, …
⎧1 x ∈ A
m A ( x ) := ⎨ m A ( x ) ∈ {0,1}
• Modeling of imprecise dependencies (e.g. rules): ⎩0 x ∉ A
– If Temperature is low and Oil is cheap then crank up the heating system
• Origin of Information: • Example: A = {x | a ≤ x ≤ b}

– Modeling of Expert Knowledge
– Representation of information extracted from inherently imprecise data
m A ( x)
1
a b x
22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005. #3 22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005. #4
Characteristic Functions: Fuzzy Sets Linguistic Variables and Values

• Fuzzy Sets are described by a membership function: • Associating meaning (semantic) with fuzzy sets results in
– Linguistic Variables: the (labeled!) domain of the fuzzy sets
µ A~ ( x ) ∈ [0,1] – Linguistic Values: a (labeled!) collection of fuzzy sets on this domain
• Example: • Examples:
– Age: young, old
A = x is roughly in [a, b]
~ µ young/old ( x) µ young µ old
1
30 50 x [years]
µ A~ ( x) – Size: small, medium, tall
1
µsml/med/tall ( x) µsmall µ medium µ tall
1
x 150 190 x [cm]

a b
M. Berthold, Uni Konstanz:Tutorial

"Fuzzy Logic", ACAI 2005. 1
Linguistic Values & Context Types of Membership Functions
• what about basketball players?
Trapezoid: <a,b,c,d> Gaussian: N(m,s)
µsml/med/tall ( x) µsmall µ medium µ tall

µ(x) µ(x)
1 1 1
s
180 220 x [cm]
0 a b c d x 0 m x
• …and jockeys?
Triangular: <a,b,b,d> Singleton: (a,1) and (b,0.5)
µsml/med/tall ( x) µsmall µ medium µ tall µ(x) µ(x)
1
1 1
140 160 x [cm]
⇒ linguistic values are inherently context dependent! 0 a b d x 0 a b x
Fuzzy Membership Function: Basic Concepts Operators on Fuzzy Sets

• Support: elements having non-zero degree of membership • Set of old and tall people (Conjunction)
µ old ( x) = 0.7
• Core: set with elements having degree of 1 µ old ∧ tall ( x) = 0.7
µ tall ( x) = 0.5
µ old ∧ tall ( x) = 0.5
• α-Cut: set of elements with degree >= α µ old ∧ tall ( x) = ?
µ tall µ old ∧ tall ( x) = 1.0
• Height: maximum degree of membership 1
µ old ∧ tall ( x) = 0.2

µA µ old ∧ tall ( x) = 0
core
1
α α-cut
height 0
0 0 1 µ old
support x
Operators on Fuzzy Sets Min / Max-Norm

• Set of old or tall people (Disjunction) • Classical Fuzzy Operators: Min/Max-Norm
– Conjunction:
µ old ( x) = 0.7 µ A∧ B ( x) := min{µ A ( x), µ B ( x)}
µ tall ( x) = 0.5 – Disjunction:
µ A∨ B ( x) := max{µ A ( x), µ B ( x)}
µ old ∨ tall ( x) = ? µ old ∨ tall ( x) = 0.7
– Negation:
µ tall µ old ∨ tall ( x) = 1.0 µ ¬A ( x) := 1 − µ A ( x)
µ old ∨ tall ( x) = 0.5 1
µ A (x) µ B (x)
1
µ old ∨ tall ( x) = 0.2
0
µ old ∨ tall ( x) = 0 x
0
0 1 µ old

Product / Bounded-Sum Lukasiewicz Norm
• Classical Fuzzy Operators: Product / Bounded-Sum
– Conjunction:
µ A∧ B ( x) := µ A ( x) ⋅ µ B ( x)
– Disjunction:
µ A∨ B ( x) := µ A ( x) + µ B ( x) − µ A ( x) ⋅ µ B ( x)
– Negation:
µ ¬A ( x) := 1 − µ A ( x)
µ A (x) µ B (x)
1
0
x
T-norms and S-norms Fuzzy T- and S-Norms

T-Norm S-Norm
S (a, b ) = max{a, b}
T (a, b ) = min{a, b}
S(u,v) = 1 – T(1-u,1-v) using De Morgans Law
T(u,v) = 1 – S(1-u,1-v) (A∧B = ¬(¬A∨¬B)) Min-Max
min/max: max(u,v) = 1 – min(1-u,1-v)

…= 1-1-min(-u,-v) = max(u,v) S (a, b ) = a + b − a ⋅ b
T (a, b ) = a ⋅ b
Product-Sum
Fuzzy T- and S-Norms Operator - Spectrum

T-Norm S-Norm
T (a, b ) = max{0, a + b − 1}
S (a, b ) = min{1, a + b} S-norms
T-norms (T-conorms)
Lukasiewicz
Norm
drastic bounded bounded drastic

min max
product product sum sum
⎧a , if b = 0
⎪
S (a, b ) = ⎨b , if a = 0
⎧a , if b = 1
{
⎪1 , else
⎪
T (a, b ) = ⎨b , if a = 1 ⎩ u if v=1
⎪0 , else
Drastic ⎩ drastic product : T(u,v)= v if u=1
Product/Sum 0 otherwise
{
u if v=0
drastic sum : S(u,v)= v if u=0
1 otherwise

Fuzzy Norms: Issues… Fuzzy Implication
• Interesting effects: • One possibility:
– A ∧ ¬A = ? – derive Implication via tautology „A→B = ¬A∨(A∧B)“ and min/max norm.
µ A (x) µ ¬A (x)
1 µ A→ B ( x) := max{1 − µ A ( x), min{µ A ( x), µ B ( x)}}
*v* ¬A A∧B
µ A∧¬A (x)
0
x
– A ∨ ¬A = ?
µ A (x) µ A∨ ¬A (x)
1
µ ¬A (x)
0
x
Fuzzy Implication Imprecise Reasoning

• Or vice versa: • Classic Modus Ponens
– start with Lukasiewicz-Implication:
A ¬A
µ A→ B ( x) := min{1,1 − µ A ( x) + µ B ( x)} A→ B A→ B
and derive disjunction and conjunction using B ?
• Imprecise:
– A∨B = ¬A→B
Generalized Modus Ponens
µ A∨ B ( x) := min{1, µ A ( x) + µ B ( x)}
µ A' ( x )
– A∧B = ¬ (¬A∨¬B) A→ B
µ B' ( y)
µ A∧ B ( x) := 1 − min{1,1 − µ A ( x) + 1 − µ B ( x)}
= max{0, µ A ( x) + µ B ( x) − 1}
Joint Constraint (support distribution) Conditional Constraint (possibility distribution)

• Using the Min/Max Norm: • Alternatively (using Lukasiewicz Norm):
Implication results in a constraint on (x,y) Implication results in relation expressing possibilities:
⇒ Cartesian Product A×B: µ A× B ( x, y ) = min{µ A ( x ), µ B ( y )} Poss( x, y ) = min{1,1 − µ A ( x) + µ B ( y )}
µ B ' ( y ) = sup{min{µ A' ( x), µ A×B ( x, y )}} µ B ' ( y ) = sup{min{µ A' ( x),1 − µ A ( x) + µ B ( x)}}
x x
y y y
µ B '' ( y ) µ B' ( y) A→ B
µ B' ( y) A× B A× B
x x µB µA x
µB µA µB µA
µ A' ( x) µ A'' ( x) µ A' ( x)
x x x

Joint vs. Conditional Constraint Fuzzy Rules
• Joint Constraints: • Rule: IF <Antecedent> THEN <Consequent>
– express positive knowledge (facts are supported) • Fuzzy Version 1: Mamdani Rules
• Conditional Constraints: – Antecedent: Conjunction of fuzzy memberships
– express negative knowledge (facts are excluded) – Consequent: Fuzzy Set
Example: Mamdani Rule Fuzzy Rules

IF age IS young AND car-power IS high THEN risk IS high • Rule: IF <Antecedent> THEN <Consequent>
• Fuzzy Version 1: Mamdani Rules
µ – Antecedent: Conjunction of fuzzy memberships
low medium high – Consequent: Fuzzy Set
risk
• Fuzzy Version 2: Takagi-Sugeno -(Kang)- Rules
age – Antecedent: Conjunction of fuzzy memberships
old – Consequent: (usually) real-valued functions of degree 0-2.
normal
young
µ µ
low medium high
car-power
Example: Takagi-Sugeno Rule Fuzzy Rule System (Mamdani)

IF age IS young AND car-power IS high R1: IF age IS young AND car-power IS high THEN risk IS high
THEN risk-factor = w0+w1*age+w2*car-power R2: IF age IS normal AND car-power IS medium THEN risk IS medium
µ µ µ
old young high high
w1*age
age
R1:
normal age car-power risk
young µ µ µ
w2*car-power normal medium medium
µ risk-factor
R2:
w0
age car-power risk
µ
low medium high
car-power

Fuzzy Rule Systems: Example 1 Fuzzy Rule Systems: Example 1
• Step 1: Fuzzification of crisp inputs • Step 2a: Inference (e.g. via Min/Max-Norm)
µ µ µ µ µ
µ R1 = 0.4 µ
0.8
R1: young normal old
0.4 low medium high low medium high R1: young normal old low medium high low medium high
age car-power risk age car-power risk

µ µ µ µ µ µ R = 0.3 µ
2
0.7
R2: young normal old low medium high low medium high R2: young normal old low medium high low medium high
0.3
µ
low medium high
age car-power age car-power risk
a b a b
Fuzzy Rule Systems: Example 1 Fuzzy Rule Systems: Example 1

• Step 2b: Inference (e.g. via Maximum-Norm) • Step 3: Defuzzification
µ µ µ µ µ µ

µ µ µ µ µ µ

µ µ
low medium high low medium high
a b a b c=?
Defuzzification: Center of Gravity Defuzzification: Other Methods

• Center of Gravity:
y=
∫ y⋅µ risk ( y ) dy
∫µ risk ( y ) dy
• An easier approximation (with some imprecision…):

r
∑µ
j =1
j ⋅sj
y= r
∑µ
j =1
j

Fuzzy Inference (Mamdani) Fuzzy Rule System (Mamdani)
• Fuzzy Inference R1: if x is small then y is medium
R2: if x is medium then y is large
y R3: if x is large then y is zero
Fuzzification Inference Defuzzification
large
Fuzzy Rules
if temp is cold
µcold µwarm µhot then valve is open µopen µhalf µclose medium
0.7 µcold =0.7 0.7

if temp is warm small
0.2 then valve is half 0.2
µwarm =0.2
Measurement t v
zero
if temp is hot Output µ µ x
then valve is close
µhot =0.0
small medium large x
Fuzzy Rule System (Takagi Sugeno) Construction of Fuzzy Rule Systems

R1: IF x IS small THEN y = x • Wang&Mendel Algorithm R1: if x is zero then y is medium
R2: if x is small then y is medium
R2: IF x IS medium THEN y = 5
R3: if x is medium then y is large
R3: IF x IS large THEN y = 2*x-5 y R4: if x is large then y is medium
large
r
r
∑µ ⋅ yi ( x )
y
Ri
y= i =1
r
medium
∑µ
5
Ri
i =1
small
x
µ
zero
small medium large µ µ x
5 x
10 zero small medium large x
Grid Based Algorithms Free Fuzzy Rules

• Exponentially many rules in high dimensional spaces • No global dependence on granulation:
• If grid is chosen fine enough: arbitrarily good approximation – individual (per rule) membership functions
possible (but at what cost!) – better modeling of local properties
• Wrong choice of grid: skipping of extrema • Not all attributes used for all rules:
• One extension: Higgins&Goodman Algorithm (see IDA-book) – individual choice of constraints on few attributes per rule
– pre-define threshold for desired approximation error – better interpretability in high dimensions
– finds “best” partitioning (=grid) – no exponential explosion of rules with dimensionality
– Disadvantages:
• concentrates on outliers
• interpretation difficult: granulation solely data driven
• Other Constructive Algorithms (coming up next…)

– Local membership functions
– partially predefined membership functions/granulations possible
– somewhat tolerant against outliers through local caching

Formation of Free Fuzzy Rules Formation of Free Fuzzy Rules
Algorithm FRL Algorithmus FRL
1. FORALL training examples (x,c) DO Steps:

1. IF correct rule of class c exists: • COVERED: easy
• COVERED:
• COMMIT: easy.
– increase weight +1
• SHRINK: heuristic
– adjust core region of rule to cover x
one solution: volume-based.
2. ELSE:
• COMMIT:
Demo.
– insert new rule with core=x
– Support = infinite (i.e. rule is not constrained)
3. SHRINK:
• Reduce Support of all Rules of conflicting class that cover x.
2. Until no more changes occured.
Formation of Free Fuzzy Rules Other Types of Fuzzy Rule Learning Methods
Observations: • Constructive:
• FRL finds set of rules completely describing the data – Finding fuzzy rules by growing from singletons
(as long as training data is conflict free…) (⇔ FRL shrinks from most general until it fits to data)
• Each rule is a partial hypothesis for a subset of training data • GRID:

– merge grid cells (or rows/columns) if no points covered or same class
– core: most specific hypothesis covering subset of training data
predicted
– support: (one of the) most general hypotheses covering subset of data.
– support is_more_general_than core.
• Adaptive:
– Initialize rules randomly (or via expert knowledge) and optimize rule
parameters (location, sometime also number of membership functions)
• Core and Support regions can also be seen as: iteratively (gradient descent, heuristic hill climbing algorithms).
– smallest area with highest degree of confidence (we have evidence) • Neuro-Fuzzy (later…):
– largest area without conflict (we have not seen any counter-examples) – inject fuzzy rules into neural network and use NN training algorithms.
Fuzzy Numbers: Motivation Fuzzy Numbers

• Often it is desirable to preprocess imprecise inputs • Classic (“crisp”) Numbers: a∈R
– add or multiply fuzzy numbers – Characteristic Function ma(x)∈{0,1}
– apply function (normalization, …) to fuzzy numbers
• Fuzzy Controller can be extended to process imprecise inputs ma
via use of fuzzy numbers.
• Fuzzy Numbers: Model Imprecision in e.g. measurement. x
• Fuzzy Number („about a“):

– membership function µa(x)∈[0,1] Membership functions
of fuzzy numbers are
µa (usually):
• normalized
• monotone (left/right)
• singular
x

Operations on Fuzzy Numbers Operations on Fuzzy Numbers
• Addition of two crisp Numbers: • Addition of two Singletons:
ma/b µa/b
1 1
a b a+b x a b a+b x
⎧1 z = a + b ⎧min{µ a (a ), µb (b)} z = a + b
ma +b ( z ) = ⎨ µ a +b ( z ) = ⎨
⎩0 else ⎩ 0 else
ma +b ( z ) = max{ma ( x ) ⋅ mb ( y ) | x + y = z} µ a +b ( z ) = max{min{µ a (x ), µb ( y )}| x + y = z}

x , y∈R x , y∈R
= and
= “and”?...

• Addition of two (fuzzy) intervals: • And finally: Addition of two Fuzzy Numbers:
µa/b µa/b
1 1
a b a+b x a b x
µ a +b ( z ) = max{min{µ a (x ), µb ( y )}| x + y = z} µ a +b ( z ) = max{min{µ a (x ), µb ( y )}| x + y = z}

x , y∈R x , y∈R
= and
what about other operators, for example multiplication?

• Multiplication of two Fuzzy Numbers: • Crisp Functions applied to Fuzzy Numbers:
µa/b
µ f ( a ) ( y ) = max{µ a ( x) | y = f ( x)}
x∈R
y
1
y = f (x )
a b x
µ a*b ( z ) = max{min{µ a (x ), µb ( y )}| x * y = z}

x , y∈R
µf(a) x
µa
what about arbitrary functions?
a x

• Crisp (non monotone) function applied to Fuzzy Numbers: • Based on Extension Principle:
µ f ( a ) ( y ) = max{µ a ( x) | y = f ( x)}
x∈R µ f ( a ) ( y ) = max {min{µ a ( x1 ),..., µ a ( xn )}| y = f ( x1 ,..., xn )}
y x1 ,..., xn ∈R 1 n
y = f (x )
• Computable in Practice:
– via series of α-cuts
– via polynomial representation of left and right side of fuzzy numbers
(closed under addition and multiplication a.o.)
– via granulation on variables (grid based)
– …
µf(a) x
µa
a x
Imprecise Functions (Fuzzy Graphs) Imprecise Functions (Fuzzy Graphs)

• Real-valued Function: infinite set of crisp points • Each fuzzy point describes an imprecise relation:
• Fuzzy Graph: finite set of imprecise points (xr, y ) IS A× B
– IF x IS small THEN y IS medium
– IF x IS medium THEN y IS small
• A fuzzy graph is a collection of fuzzy points:
– IF x IS large THEN y IS large
(xr, y ) IS A1 × B1 OR ... OR A r × Br
y
large • Reasoning using fuzzy graphs:

r
x IS A
medium ~ r
f IS U j=1 A j × B j
small
y IS B
µ µ
small medium large
x
Fuzzy Graphs Fuzzy Graphs

• Approximation through cylindrical extension: • Learning of Fuzzy Graphs from Data: Demo
{
B = proj y (A × I ) ∩ f
~
} – Influence of granulation of target variable
– Influence of outliers: large number of fuzzy points
⇒ avoidance via outlier model
1
1 A
X

Wrap-Up / Conclusions
• Fuzzy Methods
– model imprecise knowledge (fuzzy rules)
– draw imprecise conclusions
• Main Concepts:
– Fuzzy Set and Degree of Membership
– Fuzzy Set Operators
– Extension Principle: allow to extend classical functions to fuzzy numbers
• Algorithms (mostly heuristic) to
– find fuzzy rules in data
– build fuzzy approximators (fuzzy graphs/rule systems)
• Fuzzy Controllers in wide spread use
22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005. #61


Acai05 Berthold FL PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Acai05 Berthold FL PDF

Uploaded by

Copyright:

Available Formats

ALTANA Chair for Overview

Bioinformatics and Information Mining

22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005. #2

Fuzzy Logic: Motivation Characteristic Functions: Crisp Sets

• Origin of Information: • Example: A = {x | a ≤ x ≤ b}

Characteristic Functions: Fuzzy Sets Linguistic Variables and Values

x 150 190 x [cm]

M. Berthold, Uni Konstanz:Tutorial

µsml/med/tall ( x) µsmall µ medium µ tall

Fuzzy Membership Function: Basic Concepts Operators on Fuzzy Sets

µ old ∧ tall ( x) = 0.2

Operators on Fuzzy Sets Min / Max-Norm

M. Berthold, Uni Konstanz:Tutorial

T-norms and S-norms Fuzzy T- and S-Norms

min/max: max(u,v) = 1 – min(1-u,1-v)

Fuzzy T- and S-Norms Operator - Spectrum

drastic bounded bounded drastic

M. Berthold, Uni Konstanz:Tutorial

Fuzzy Implication Imprecise Reasoning

Joint Constraint (support distribution) Conditional Constraint (possibility distribution)

M. Berthold, Uni Konstanz:Tutorial

Example: Mamdani Rule Fuzzy Rules

Example: Takagi-Sugeno Rule Fuzzy Rule System (Mamdani)

M. Berthold, Uni Konstanz:Tutorial

age car-power risk age car-power risk

low medium high

age car-power age car-power risk

Fuzzy Rule Systems: Example 1 Fuzzy Rule Systems: Example 1

age car-power risk age car-power risk

age car-power risk age car-power risk

low medium high low medium high

age car-power risk age car-power risk

Defuzzification: Center of Gravity Defuzzification: Other Methods

• An easier approximation (with some imprecision…):

M. Berthold, Uni Konstanz:Tutorial

0.7 µcold =0.7 0.7

Fuzzy Rule System (Takagi Sugeno) Construction of Fuzzy Rule Systems

Grid Based Algorithms Free Fuzzy Rules

• Other Constructive Algorithms (coming up next…)

M. Berthold, Uni Konstanz:Tutorial

1. FORALL training examples (x,c) DO Steps:

• Each rule is a partial hypothesis for a subset of training data • GRID:

Fuzzy Numbers: Motivation Fuzzy Numbers

via use of fuzzy numbers.

• Fuzzy Numbers: Model Imprecision in e.g. measurement. x

• Fuzzy Number („about a“):

M. Berthold, Uni Konstanz:Tutorial

ma +b ( z ) = max{ma ( x ) ⋅ mb ( y ) | x + y = z} µ a +b ( z ) = max{min{µ a (x ), µb ( y )}| x + y = z}

Operations on Fuzzy Numbers Operations on Fuzzy Numbers

µ a +b ( z ) = max{min{µ a (x ), µb ( y )}| x + y = z} µ a +b ( z ) = max{min{µ a (x ), µb ( y )}| x + y = z}

what about other operators, for example multiplication?

Operations on Fuzzy Numbers Operations on Fuzzy Numbers

µ a*b ( z ) = max{min{µ a (x ), µb ( y )}| x * y = z}

M. Berthold, Uni Konstanz:Tutorial

Imprecise Functions (Fuzzy Graphs) Imprecise Functions (Fuzzy Graphs)

large • Reasoning using fuzzy graphs:

Fuzzy Graphs Fuzzy Graphs

M. Berthold, Uni Konstanz:Tutorial

22/06/2005 M. Berthold, Uni Konstanz:Tutorial "Fuzzy Logic", ACAI 2005. #61

M. Berthold, Uni Konstanz:Tutorial

You might also like

µ ab ( z ) = max{min{µ a (x ), µb ( y )}| x y = z}