Professional Documents
Culture Documents
Chapter 6
Association Analysis: Advance Concepts
Different methods:
– Discretization-based
– Statistics-based
– Non-discretization based
minApriori
Unsupervised:
<1 2 3> <4 5 6> <7 8 9>
– Equal-width binning
– Equal-depth binning <1 2 > <3 4 5 6 7 > < 8 9>
– Cluster-based
Supervised discretization
Continuous attribute, v
1 2 3 4 5 6 7 8 9
Interval width
Number of intervals = k
Total number of Adjacent intervals = k(k-1)/2
Execution time
– If the range is partitioned into k intervals, there are O(k 2) new items
– If an interval [a,b) is frequent, then all intervals that subsume [a,b)
must also be frequent
E.g.: if {Age [21,25), Chat Online=Yes} is frequent,
then {Age [10,50), Chat Online=Yes} is also frequent
– Improve efficiency:
Use maximum support to avoid intervals that are too wide
Redundant rules
Example:
{Income > 100K, Online Banking=Yes} Age: =34
Rule consequent consists of a continuous variable,
characterized by their statistics
– mean, median, standard deviation, etc.
Approach:
– Withhold the target attribute from the rest of the data
– Extract frequent itemsets from the rest of the attributes
Binarized the continuous attributes (except for the target attribute)
– For each frequent itemset, compute the corresponding
descriptive statistics of the target attribute
Frequent itemset becomes a rule by introducing the target variable
as rule consequent
– Apply statistical test to determine interestingness of the rule
'
– Statistical hypothesis testing: Z
s12 s22
Null hypothesis: H0: ’ = +
n1 n2
Alternative hypothesis: H1: ’ > +
Z has zero mean and variance 1 under null hypothesis
Example:
r: Browser=Mozilla Buy=Yes Age: =23
– Rule is interesting if difference between and ’ is more than 5
years (i.e., = 5)
– For r, suppose n1 = 50, s1 = 3.5
– For r’ (complement): n2 = 250, s2 = 6.5
' 30 23 5
Z 3.11
2 2 2 2
s s 3.5 6.5
1
2
n1 n2 50 250
Document-term matrix:
TID W 1 W 2 W 3 W 4 W 5
D1 2 2 0 0 1
D2 0 0 1 2 2
D3 2 3 0 0 0
D4 0 0 1 0 1
D5 1 1 1 0 2
Example:
W1 and W2 tends to appear together in the
same document
TID W 1 W 2 W 3 W 4 W 5 TID W1 W2 W3 W4 W5
D1 2 2 0 0 1 D1 0.40 0.33 0.00 0.00 0.17
Normalize
D2 0 0 1 2 2 D2 0.00 0.00 0.33 1.00 0.33
D3 2 3 0 0 0 D3 0.40 0.50 0.00 0.00 0.00
D4 0 0 1 0 1 D4 0.00 0.00 0.33 0.00 0.17
D5 1 1 1 0 2 D5 0.20 0.17 0.33 0.00 0.33
TID W1 W2 W3 W4 W5 Example:
D1 0.40 0.33 0.00 0.00 0.17
Sup(W1,W2,W3)
D2 0.00 0.00 0.33 1.00 0.33
D3 0.40 0.50 0.00 0.00 0.00 = 0 + 0 + 0 + 0 + 0.17
D4 0.00 0.00 0.33 0.00 0.17 = 0.17
D5 0.20 0.17 0.33 0.00 0.33
TID W1 W2 W3 W4 W5
D1 0.40 0.33 0.00 0.00 0.17
D2 0.00 0.00 0.33 1.00 0.33
D3 0.40 0.50 0.00 0.00 0.00
D4 0.00 0.00 0.33 0.00 0.17
D5 0.20 0.17 0.33 0.00 0.33
Example:
Sup(W1) = 0.4 + 0 + 0.4 + 0 + 0.2 = 1
Sup(W1, W2) = 0.33 + 0 + 0.4 + 0 + 0.17 = 0.9
Sup(W1, W2, W3) = 0 + 0 + 0 + 0 + 0.17 = 0.17
Food
Electronics
Bread
Foremost Kemps
Printer Scanner
Approach 1:
– Extend current association rule formulation by augmenting each
transaction with higher level items
Issues:
– Items that reside at higher levels have much higher support
counts
if support threshold is low, too many frequent patterns involving items
from the higher levels
– Increased dimensionality of the data
02/14/2018 Introduction to Data Mining, 2nd Edition 27
Multi-level Association Rules
Approach 2:
– Generate frequent patterns at highest level first
Issues:
– I/O requirements will increase dramatically because
we need to perform more passes over the data
– May miss some potentially interesting cross-level
association patterns
Sequential Patterns
Examples of Sequence
Element
Event
(Transaction)
E1 E1 E3 (Item)
E2 E2
E2 E3 E4
Sequence
02/14/2018 Introduction to Data Mining, 2nd Edition 32
Sequence Data
Sequence Database:
Sequence ID Timestamp Events Sequence A:
A 10 2, 3, 5
A 20 6, 1
A 23 1
B 11 4, 5, 6
B 17 2 Sequence B:
B 21 7, 8, 1, 2
B 28 1, 6
C 14 1, 8, 7
Sequence C:
s: b1 b2 b3 b4 b5
t: a1 a2 a3
t is a subsequence of s if a1 b2, a2 b3, a3 b5.
Data sequence Subsequence Contain?
< {2,4} {3,5,6} {8} > < {2} {8} > Yes
< {1,2} {3,4} > < {1} {2} > No
< {2,4} {2,4} {2,5} > < {2} {4} > Yes
<{2,4} {2,5}, {4,5}> < {2} {4} {5} > No
<{2,4} {2,5}, {4,5}> < {2} {5} {5} > Yes
<{2,4} {2,5}, {4,5}> < {2, 4, 5} > No
Given:
– a database of sequences
– a user-specified minimum support threshold, minsup
Task:
– Find all subsequences with support ≥ minsup
Candidate 2-subsequences:
<{i1, i2}>, <{i1, i3}>, …,
<{i1} {i1}>, <{i1} {i2}>, …, <{in} {in}>
Candidate 3-subsequences:
<{i1, i2 , i3}>, <{i1, i2 , i4}>, …,
<{i1, i2} {i1}>, <{i1, i2} {i2}>, …,
<{i1} {i1 , i2}>, <{i1} {i1 , i3}>, …,
<{i1} {i1} {i1}>, <{i1} {i1} {i2}>, …
()
Given 2 events: a, b
(a) (b)
Candidate 1-subsequences: (a,b)
<{a}>, <{b}>.
Item-set patterns
Candidate 2-subsequences:
<{a} {a}>, <{a} {b}>, <{b} {a}>, <{b} {b}>, <{a, b}>.
Candidate 3-subsequences:
<{a} {a} {a}>, <{a} {a} {b}>, <{a} {b} {a}>, <{a} {b} {b}>,
<{b} {b} {b}>, <{b} {b} {a}>, <{b} {a} {b}>, <{b} {a} {a}>
<{a, b} {a}>, <{a, b} {b}>, <{a} {a, b}>, <{b} {a, b}>
Step 1:
– Make the first pass over the sequence database D to yield all the 1-
element frequent sequences
Step 2:
Repeat until no new frequent sequences are found
– Candidate Generation:
Merge pairs of frequent subsequences found in the (k-1)th pass to generate
candidate sequences that contain k items
– Candidate Pruning:
Prune candidate k-sequences that contain infrequent (k-1)-subsequences
– Support Counting:
Make a new pass over the sequence database D to find the support for these
candidate sequences
– Candidate Elimination:
Eliminate candidate k-sequences whose actual support is less than minsup
Frequent
3-sequences
Frequent
3-sequences
xg = 2, ng = 0, ms= 4
Approach 1:
– Mine sequential patterns without timing constraints
– Postprocess the discovered patterns
Approach 2:
– Modify GSP to directly prune candidates that violate
timing constraints
– Question:
Does Apriori principle still hold?
s is a contiguous subsequence of
w = <e1>< e2>…< ek>
if any of the following conditions hold:
1. s is obtained from w by deleting an item from either e1 or ek
2. s is obtained from w by deleting an item from any element ei
that contains at least 2 items
3. s is a contiguous subsequence of s’ and s’ is a contiguous
subsequence of w (recursive definition)
xg: max-gap
{A B} {C} {D E}
ng: min-gap
<= xg >ng <= ws
ws: window size
<= ms
ms: maximum span
xg = 2, ng = 0, ws = 1, ms= 5
< {1} {2} {3} {4} {5}> < {1,2} {3,4} > No
< {1,2} {2,3} {3,4} {4,5}> < {1,2} {3,4} > Yes
<… {a c} … >,
<… {a} … {c}…> ( where time({c}) – time({a}) ≤ ws)
<…{c} … {a} …> (where time({a}) – time({c}) ≤ ws)
E1 E3 E1 E1 E2 E4 E1 E2 E1
E2 E4 E2 E3 E4 E2 E3 E5 E2 E3 E5 E2 E3 E1
Assume:
xg = 2 (max-gap)
ng = 0 (min-gap)
ws = 0 (window size)
ms = 2 (maximum span)
Subgraph Mining
Frequent Subgraph Mining
Research
Artificial Databases
Intelligence
Data Mining
a a a
q p p p
p
b a a a
s s s
r r
p t t t t
r r
c c c c
p q p p
b b b
TID = 1: A
Transaction
Items
Id
C
1 {A,B,C,D}
2 {A,B,E} B
3 {B,C}
4 {A,B,D,E}
5 {B,C,D}
E
D
a a e
q
e p
p p
q p q
r b c d
b b
d
r r r
p r
c d a
G1 G2 G3
Support:
– number of graphs that contain a particular subgraph
In Apriori:
– Merging two frequent k-itemsets will produce a
candidate (k+1)-itemset
b
e + b
e
b
e
c c c
e
a a b
b
e + b
e
c
a
c c e
e
b
c
+
a a a b a a
b a a b a b a
v1 v2 v1 v2
p p
a a a a v1 v2 v3 v4
p p p
p a b b a
p p p p
p
a a a a b v5
v3 p v4 v3 p v4
G1 G2 G3
Core Core
Case 1: a c and b d
G3 = Merge(G1,G2)
a b
Core c d
Case 2: a = c and b d
G3 = Merge(G1,G2) G3 = Merge(G1,G2)
a b a b
Core a d Core d
Case 3: a c and b = d
G3 = Merge(G1,G2) G3 = Merge(G1,G2)
a b a b
Core a d Core c
Case 4: a = c and b = d
G3 = Merge(G1,G2)
a b
Core a b
G3 = Merge(G1,G2) G3 = Merge(G1,G2)
a b a b
Core b Core a
A(2) A(1) A(1) A(2) A(3) A(4) B(5) B(6) B(7) B(8)
A(1) 1 1 0 1 0 1 0 0
B (7) B (6) A(2) 1 1 1 0 0 0 1 0
A(3) 0 1 1 1 1 0 0 0
A(4) 1 0 1 1 0 0 0 1
B (5) B (8) B(5) 0 0 1 0 1 0 1 1
B(6) 1 0 0 0 0 1 1 1
B(7) 0 1 0 0 1 1 1 0
A(3) A(4) B(8) 0 0 0 1 1 1 0 1
• The same graph can be represented in many ways
0 0 1 0 0 1 1 1
0 0 1 1 1 0 1 0
1 1 0 1 1 1 0 0
0 1 1 0 1 0 0 0
String: 011011 Canonical: 111100
0 0 0 e1 e0 e0 > 0 0 0 e0 e1 e0
(Canonical Label)