Data Mining

BITS Pilani
Pilani|Dubai|Goa|Hyderabad M5: Association Analysis
 BITS Pilani
Pilani|Dubai|Goa|Hyderabad

5.1 Association Analysis Concepts

Source Courtesy: Some of the contents of this PPT are sourced from materials provided by publishers of prescribed books
 Association Analysis

• Association analysis measures the strength of co-occurrence between one item

and another. The objective of this class of data mining algorithms is not to predict
an occurrence of an item, like classification or regression do, but to find usable
patterns in the co-occurrences of the items. Association rules learning is a branch
of an unsupervised learning process that discovers hidden patterns in data, in the
form of easily recognizable rules
• Association algorithms are widely used in retail analysis of transactions,
recommendation engines, and online clickstream analysis across web pages. One
of the popular applications of this technique is called market basket analysis,
which finds co-occurrences of one retail item with another item within the same
retail purchase transaction
• retailer can take advantage of this association for bundle pricing, product
placement, and even shelf space optimization within the store layout.

Data Mining
1/30/22
3
BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956
 Association Rule Mining

• Given a set of transactions, find rules that will predict the

occurrence of an item based on the occurrences of other
items in the transaction
Market-Basket transactions
Example of Association Rules
TID Items
{Diaper}  {Butter},
1 Bread, Milk
{Milk, Bread}  {Eggs, Coke},
2 Bread, Diaper, Butter, Eggs {Butter, Bread}  {Milk},
3 Milk, Diaper, Butter, Coke
4 Bread, Milk, Diaper, Butter
5 Bread, Milk, Diaper, Coke Implication means co-occurrence,
not causality!

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Definition: Frequent Itemset

• Itemset TID Items

• A collection of one or more items 1 Bread, Milk
• Example: {Milk, Bread, Diaper} 2 Bread, Diaper, Butter, Eggs
• k-itemset 3 Milk, Diaper, Butter, Coke
• An itemset that contains k items 4 Bread, Milk, Diaper, Butter
• Support count () 5 Bread, Milk, Diaper, Coke
• Frequency of occurrence of an itemset
• E.g. ({Milk, Bread,Diaper}) = 2
• Support
• Fraction of transactions that contain an itemset
• E.g. s({Milk, Bread, Diaper}) = 2/5
• Frequent Itemset
• An itemset whose support is greater than or equal to a minsup threshold

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Definition: Association Rule
TID Items
 Association Rule 1 Bread, Milk
– An implication expression of the form 2 Bread, Diaper, Butter, Eggs
X  Y, where X and Y are itemsets 3 Milk, Diaper, Butter, Coke
– Example: 4 Bread, Milk, Diaper, Butter
{Milk, Diaper}  {Butter} 5 Bread, Milk, Diaper, Coke

 Rule Evaluation Metrics

– Support (s) Example:
 Fraction of transactions that contain {Milk, Diaper}  Butter
both X and Y
 (Milk, Diaper, Butter) 2
– Confidence (c) s   0.4
|T| 5
 Measures how often items in Y
appear in transactions that  (Milk, Diaper, Butter ) 2
contain X c   0.67
 ( Milk, Diaper) 3

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Association Rule Mining Task

• Given a set of transactions T, the goal of association rule mining is to

find all rules having
• support ≥ minsup threshold
• confidence ≥ minconf threshold

• Brute-force approach:
• List all possible association rules
• Compute the support and confidence for each rule
• Prune rules that fail the minsup and minconf thresholds
 Computationally prohibitive!

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Mining Association Rules
TID Items
1 Bread, Milk
2 Bread, Diaper, Butter, Eggs
3 Milk, Diaper, Butter, Coke
4 Bread, Milk, Diaper, Butter
5 Bread, Milk, Diaper, Coke
Example of Rules:
{Milk,Diaper}  {Butter} (s=0.4, c=0.67)
{Milk,Butter}  {Diaper} (s=0.4, c=1.0)
{Diaper,Butter}  {Milk} (s=0.4, c=0.67)
{Butter}  {Milk,Diaper} (s=0.4, c=0.67)
{Diaper}  {Milk,Butter} (s=0.4, c=0.5)
{Milk}  {Diaper,Butter} (s=0.4, c=0.5)

Observations:
• All the above rules are binary partitions of the same itemset:
{Milk, Diaper, Butter}
• Rules originating from the same itemset have identical support but
can have different confidence
• Thus, we may decouple the support and confidence requirements

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Mining Association Rules

• Two-step approach:
1. Frequent Itemset Generation
– Generate all itemsets whose support  minsup

2. Rule Generation
– Generate high confidence rules from each frequent itemset, where each rule is a
binary partitioning of a frequent itemset

• Frequent itemset generation is still computationally expensive

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

 Frequent Itemset Generation
null

A B C D E

AB AC AD AE BC BD BE CD CE DE

ABC ABD ABE ACD ACE ADE BCD BCE BDE CDE

ABCD ABCE ABDE ACDE BCDE

Given d items, there
are 2d possible
ABCDE candidate itemsets
Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Frequent Itemset Generation
• Brute-force approach:
• Each itemset in the lattice is a candidate frequent itemset
• Count the support of each candidate by scanning the database
• Match each transaction against every candidate
• Complexity ~ O(NMw) => Expensive since M = 2d !!!
Transactions List of
Candidates
TID Items
1 Bread, Milk
2 Bread, Diaper, Beer, Eggs
N 3 Milk, Diaper, Beer, Coke M
4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke
w

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Frequent Itemset Generation Strategies

• Reduce the number of candidates (M)

• Complete search: M=2d
• Use pruning techniques to reduce M

• Reduce the number of transactions (N)

• Reduce size of N as the size of itemset increases
• Used by DHP and vertical-based mining algorithms

• Reduce the number of comparisons (NM)

• Use efficient data structures to store the candidates or transactions
• No need to match every candidate against every transaction

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956

Prescribed Text Books

Author(s), Title, Edition, Publishing House

T1 Tan P. N., Steinbach M & Kumar V. “Introduction to Data Mining” Pearson
Education 
T2 Data Mining: Concepts and Techniques, Third Edition by Jiawei Han,
Micheline Kamber and Jian Pei Morgan Kaufmann Publishers

R1 Predictive Analytics and Data Mining: Concepts and Practice with

RapidMiner
by Vijay Kotu and Bala Deshpande Morgan Kaufmann Publishers

Data Mining

BITS Pilani, Deemed to be University under Section 3 of UGC Act, 1956