0 views

Uploaded by nishanthshekhar

cl

save

You are on page 1of 29

Hierarchical Clustering

**Produces a set of nested clusters organized as
**

a hierarchical tree

**Can be visualized as a dendrogram
**

**A tree-like diagram that records the sequences of
**

merges or splits

5

6

0.2

4

3

0.15

4

2

5

2

0.1

1

0.05

0

3

1

3

2

5

4

6

1

g. product catalogs) etc ...g.Strengths of Hierarchical Clustering No assumptions on the number of clusters Any desired number of clusters can be obtained by ‘cutting’ the dendogram at the proper level Hierarchical clusterings may correspond to meaningful taxonomies Example in biological sciences (e. web (e. phylogeny reconstruction. etc).

all-inclusive cluster At each step.Hierarchical Clustering Two main types of hierarchical clustering Agglomerative: Start with the points as individual clusters At each step. split a cluster until each cluster contains a point (or there are k clusters) Traditional hierarchical algorithms use a similarity or distance matrix Merge or split one cluster at a time . merge the closest pair of clusters until only one cluster (or k clusters) left Divisive: Start with one.

Complexity of hierarchical clustering Distance matrix is used for deciding which clusters to merge/split At least quadratic in the number of data points Not usable for large datasets .

4. 6. 3.Agglomerative clustering algorithm Most popular hierarchical clustering technique Basic algorithm 1. 2. 5. Compute the distance matrix between the input data points Let each data point be a cluster Repeat Merge the two closest clusters Update the distance matrix Until only a single cluster remains Key operation is the computation of the distance between two clusters Different definitions of the distance between clusters lead to different algorithms .

Input/ Initial setting Start with clusters of individual points and a distance/proximity matrix p1 p2 p3 p4 p5 . p1 p2 p3 p4 p5 . . . Distance/Proximity Matrix ...

Intermediate State After some merging steps. we have some clusters C1 C2 C3 C4 C5 C1 C2 C3 C4 C3 C4 C5 C1 Distance/Proximity Matrix C2 C5 .

Intermediate State Merge the two closest clusters (C2 and C5) and update the distance matrix. C1 C2 C3 C4 C5 C1 C2 C3 C4 C3 C4 C5 C1 Distance/Proximity Matrix C2 C5 .

After Merging “How do we update the distance matrix?” C1 C1 C3 C4 C1 C2 U C5 C2 U C5 C2 U C5 C3 C4 ? ? ? ? ? C3 ? C4 ? .

Distance between two clusters Each cluster is a set of points How do we define distance between two sets of points Lots of alternatives Not an easy task .

Distance between two clusters Single-link distance between clusters Ci and Cj is the minimum distance between any object in Ci and any object in Cj The distance is defined by the two most similar objects Dsl Ci . y ) x Ci . C j min x . y C j . y d ( x.

40 1.90 0.60 0.65 0. i.10 0.e. I1 I2 I3 I4 I5 I1 1.40 0.00 1 2 3 4 5 .20 I2 0.80 I5 0.30 0.65 0.10 0.Single-link clustering: example Determined by one pair of points..70 0.00 0.90 1.00 0.60 0.80 1. by one link in the proximity graph.70 1.50 I3 0.00 0.50 0.20 0.30 I4 0.00 0.

2 2 1 2 3 4 4 Nested Clusters 0.15 6 0.05 0 3 6 2 5 Dendrogram 4 1 .Single-link clustering: example 5 1 3 5 0.1 0.

Strengths of single-link clustering Original Points • Can handle non-elliptical shapes Two Clusters .

elongated clusters Two Clusters .Limitations of single-link clustering Original Points • Sensitive to noise and outliers • It produces long.

y C j . y d ( x. y ) x Ci .Distance between two clusters Complete-link distance between clusters Ci and Cj is the maximum distance between any object in Ci and any object in Cj The distance is defined by the two most dissimilar objects Dcl Ci . C j max x .

20 I2 0.70 0.60 0.30 I4 0.50 I3 0.00 1 2 3 4 5 .30 0.90 0.00 0.00 0.50 0.Complete-link clustering: example Distance between clusters is determined by the two most distant points in the different clusters I1 I2 I3 I4 I5 I1 1.00 0.90 1.40 1.10 0.00 0.70 1.80 I5 0.10 0.65 0.65 0.80 1.40 0.20 0.60 0.

Complete-link clustering: example 4 1 2 5 5 0.25 3 3 6 1 4 Nested Clusters 0.4 0.2 0.15 0.3 0.05 0 3 6 4 Dendrogram 1 2 5 .35 2 0.1 0.

Strengths of complete-link clustering Original Points Two Clusters • More balanced clusters (with equal diameter) • Less susceptible to noise .

Limitations of complete-link clustering Original Points Two Clusters • Tends to break large clusters • All clusters tend to have the same diameter – small clusters are merged with larger ones .

y ) xCi .Distance between two clusters Group average distance between clusters Ci and Cj is the average distance between any object in Ci and any object in Cj 1 Davg Ci . C j Ci C j d ( x. yC j .

65 0.00 0.60 0.10 0.20 I2 0.65 0.30 I4 0.00 1 2 3 4 5 .50 0. I1 I2 I3 I4 I5 I1 1.90 0.60 0.Average-link clustering: example Proximity of two clusters is the average of pairwise proximity between points in the two clusters.40 0.00 0.50 I3 0.00 0.20 0.40 1.00 0.10 0.80 1.70 1.70 0.90 1.30 0.80 I5 0.

15 3 6 1 4 3 Nested Clusters 0.1 0.25 0.2 2 0.Average-link clustering: example 5 4 1 2 5 0.05 0 3 6 4 1 Dendrogram 2 5 .

Average-link clustering: discussion Compromise between Single and Complete Link Strengths Less susceptible to noise and outliers Limitations Biased towards globular clusters .

rj ) .Distance between two clusters Centroid distance between clusters Ci and Cj is the distance between the centroid ri of Ci and the centroid rj of Cj Dcentroids Ci . C j d (ri .

C j x ri x rj x rij 2 xCi ri: centroid of Ci rj: centroid of Cj rij: centroid of Cij 2 xC j xCij 2 . and the within cluster sum of squares resulting from merging the two clusters in cluster Cij Dw Ci .Distance between two clusters Ward’s distance between clusters Ci and Cj is the difference between the total within cluster sum of squares for the two clusters separately.

Ward’s distance for clusters Similar to group average and centroid distance Less susceptible to noise and outliers Biased towards globular clusters Hierarchical analogue of k-means Can be used to initialize k-means .

Hierarchical Clustering: Comparison 5 1 3 5 2 4 6 MIN MAX 5 2 5 1 5 Ward’s Method 3 6 5 4 1 2 2 Group Average 3 1 1 4 6 4 2 3 3 3 2 4 5 4 1 5 1 2 1 3 2 4 4 3 6 .

- An Enhancement of User Behavior using Closed Set of Agglomerative Approach with GRC ConstraintUploaded byIJRASETPublications
- A survey on Efficient Enhanced K-Means Clustering AlgorithmUploaded byInternational Journal for Scientific Research and Development - IJSRD
- Similarity Sets - A New Concept of Sets to Seamlessly Handle Similarity in Database Management SystemsUploaded bydam
- A Hierarchical Clustering Algorithm Based on KUploaded byjohncena90
- SegmentationUploaded byAlbin Robin
- EFFECTIVE DATA MINING FOR PROPER MINING CLASSIFICATION USING NEURAL NETWORKSUploaded byLewis Torres
- article analysisUploaded byapi-438127639
- Paper 2Uploaded byRakeshconclave
- ROCK a Robust Clustering Algorithm for Categorical Attributes (2000)Guha00rockUploaded byabcd2003959
- CA - wikiUploaded byrudy_tanaga
- Data Mining NotesUploaded bysbr11
- An Analysis of Demographic and Benefit Sought Effects on Purchasing Outcomes in the Croatian Footwear MarketUploaded byjopa00
- Providing Query Suggestions and Ranking for User Search HistoryUploaded byRahul Sharma
- Diagnosis of Breast Cancer by Modular Evolutionary Neural NetworksUploaded byRahul Kala
- mis 441 final project reportUploaded byapi-385010636
- Cluster AnalysisUploaded byJessica Mcclain
- Implementation of Trust Issues in EcommerceUploaded byEditor IJRITCC
- Enhanced Clustering Technique for Search Engine Results using K-AlgorithmUploaded byIJSTE
- WaterNetGen an EPANET Extension for Auto (1)Uploaded bySolomon Eluma
- Data Extractionsoruabh/pdfUploaded bySaurabh Talwar
- MLUploaded byMalvikaSingh
- A Person Centered Investigation of AcademicUploaded byKyle Cayaban