8 views

Uploaded by Amit Singh

clustering

- 6867term Project
- email clustering
- Python Data Science Cookbook - Sample Chapter
- Density Based Clustering Approach for Solving the Software Component Restructuring
- Part 4
- IEEE_IP08
- 8-som with e-miner
- Real Time Video Classification i j Mir 2014
- Improved K-means clustering on Hadoop
- DataMining and Emotional Intelli - ClusterAnalysis
- Bio Medical Data Analysis Using Novel Clustering Techniques
- Educational Data Mining A Blend of Heuristic and K-Means Algorithm to Cluster Students to Predict Placement Chance
- Part 1
- A Systematic study of Text Mining Techniques
- 1101.4270.pdf
- Btech Sem6 Cs1141 Data Mining
- Data Warehousing Introduction
- Presentation 1
- segmentation
- Clustering of Vehicle Trajectories

You are on page 1of 26

António R. C. Paiva

EEL 6814 – Spring 2008

Outline

Competitive learning

Clustering

Self-Organizing Maps

• For every input the PEs will produce an output. and each broker competes by bidding with a value. given the input. The most suitable output is the highest value! . Only the “most suitable” output is utilized. consider bidding in the stock market. the PEs in a neural network will compete for the “resources. The stock are the input. • As an analogy. Only the winner PE is updated.What is competition in neural networks? • Competition means that.” such as the output.

• Competition is the base concept for clustering and self-organizing maps (SOMs). Specialization means that. through competition. the PEs are tuned for different areas of the input space. resources are limited.Why is competition necessary? • Competition creates specialization in the network. • In many situations. . so competition recreates these natural constraints in the environment.

Characteristics of competitive learning • Competitive learning is typically applied to a single-layer topology. • Competition is by itself a non-linear process and thus difﬁcult to treat mathematically. • Competitive learning is unsupervised learning. . Formulations using multi-layer topologies exist but typically employ independent competition on each layer.

y∗ = arg max x − y y (Notice that error can be deﬁned with different metrics and depends on the application.) Utilized in the formulation of clustering methods and SOM. .Criteria for competitive learning I • Error minimization Select the PE such that the output yields the minimum ‘error’.

an histogram of how many times each PE was a winner is approximately uniform. on average. Important for density estimation. all PEs are equally likely to be a winner. . Depends on the formulation.Criteria for competitive learning II • Entropy maximization Selects the PE such that. but entropy maximization can be achieved by error minimization. Put differently.

Outline Competitive learning Clustering Self-Organizing Maps .

The reference points are called centroids and each centroid deﬁnes a cluster. That is. The difference with PCA is that a cluster is a hard neighborhood. • Clustering aims at representing the input space of the data with a small number of reference points. . any point in the neighborhood of the reference point is represented by that point.Clustering • Clustering is a particular example of competitive learning. and therefore unsupervised learning.

where K is the number of clusters (or centroids).K-means • K-means is perhaps the simplest and most widely used clustering method. . • Cost function: K J= i=1 xj ∈Ci yi − xj 2 . yi is the ith centroid and Ci are the input data points within the corresponding cluster. • K-means minimizes the reconstruction MSE.

⇔ xj ∈Ck xj = xj ∈Ck yk = Nk yk ⇔ yk = 1 Nk xj ∈Ck .Optimization of K-means • Take the gradient of the cost function ∂J = ∂ yk = K 2(yi − xj ) i=1 xj ∈Ci ∂ (yi − xj ) ∂ yk 2(yk − xj ) xj ∈Ck • Setting the gradient to zero gives the ﬁxed-point update rule.

smaller error).1 Assign each data point to the “nearest” centroid (i.2 Compute new location of centroids. 2. this algorithm is known to converge to a local minima in a ﬁnite number of steps.Algorithm 1.. Initialization: Select K random data points as centroids. 2.e. While “change in J is large” 2. . Note: • For ﬁnite data.

de/ini/VDM/ research/gsn/DemoGNG/LBG_2.neuroinformatik.ruhr-uni-bochum.html .Demo From: http://www.

. • The K-means is equivalent to the “Linde-Buze-Gray” (LBG) algorithm commonly known in vector quantization.Clustering and vector quantization I • Vector quantization is an important application of clustering in engineering.

Idea: The centroids are known both by the encoder and decoder. send the index of the centroid that is nearest. At the decoder end. .Clustering and vector quantization II • Vector quantization is a form of clustering aimed at lossy data compression. use the centroid itself as an approximation to the original data data point. Instead of transmit the data.

Outline Competitive learning Clustering Self-Organizing Maps .

” Input .Self-Organizing Maps • Self-organizing maps (SOMs. • “The goal of SOM is to transform the input space into a 1-D or 2-D discrete map in a topologically ordered fashion. also known as Kohonen SOM maps) are another example of competitive learning.

Topologically preserving means that data points close in the input space are represented by nearby points in the SOM.Distinct feature • SOM builds a topologically preserving map. Self-organizing means that the competitive learning process ﬁnds this topology directly from data. .

Implementation • Each step of training involves three processes: 1. 3. . Cooperation: The winner PE deﬁnes a neighborhood PEs that are updated. Adaptation: All PEs in the neighborhood of the winner PE are adapted to optimize the criterion. weighted by the topological distance to the winner PE. 2. • Training of the SOM can be divided in two phases: ordering and convergence. Competition: The network computes the winner PE based on some criterion.

. Although there is still only one winner. • Cooperation is obtained by soft-competition.Cooperation process I • Cooperation between neighboring PEs implements lateral interaction between neurons in biological systems. a neighborhood of the winner is updated.

i. The amplitude must decrease monotonically to zero.j di2.j = exp − 0 di.j hi.e.Cooperation process II • The neighborhood function hi.j must be: Symmetric around the origin.j 2σ 2 (n) (di. This implies that the function is shift-invariant.) .. The “width” must be adjustable.j is the topological distance. • A typically choice for the neighborhood function is the Gaussian: hi. the distance in the map.

Cooperation process III • Cooperation forces neighboring PEs to tune for neighboring areas of the input space. • Cooperation is the process responsible for self-organization (and topology preservation) in SOMs. .

where i is the index of the winner PE. .j (n)(x − wi (n)).Adaptation process I • At each step the PEs are adapted according to wi (n + 1) = wi (n) + η (n)hi.

Decreasing σ means that cooperation exist at ﬁrst.Adaptation process II • At each epoch. the PEs will ﬁne tune to their speciﬁc areas. 1 + α nmax σ0 n . the stepsize η and “width” of the neighborhood function σ are reduced according to some rule. η (n) = σ (n) = η0 n . . 1 + β nmax Decreasing η creates simulated annealing. For example. but towards the end.

Demo From: http://www.neuroinformatik.html .de/ini/VDM/ research/gsn/DemoGNG/SOM.ruhr-uni-bochum.

.Applications • Visualization of higher dimensional data or process. • Density estimation.

- 6867term ProjectUploaded byjimakosjp
- email clusteringUploaded bySai Kiran Pamidi
- Python Data Science Cookbook - Sample ChapterUploaded byPackt Publishing
- Density Based Clustering Approach for Solving the Software Component RestructuringUploaded byIRJET Journal
- Part 4Uploaded byArish Qureshi
- IEEE_IP08Uploaded bysumbal_iqbal
- 8-som with e-minerUploaded bytehnic1
- Real Time Video Classification i j Mir 2014Uploaded bySai Krishna
- Improved K-means clustering on HadoopUploaded byEditor IJRITCC
- DataMining and Emotional Intelli - ClusterAnalysisUploaded byjeffconnors
- Bio Medical Data Analysis Using Novel Clustering TechniquesUploaded byHarikrishnan Shunmugam
- Educational Data Mining A Blend of Heuristic and K-Means Algorithm to Cluster Students to Predict Placement ChanceUploaded byEditor IJTSRD
- Part 1Uploaded byIrfan Aslam
- A Systematic study of Text Mining TechniquesUploaded byAnonymous qwgN0m7oO
- 1101.4270.pdfUploaded byAndika Saputra
- Btech Sem6 Cs1141 Data MiningUploaded bybhushaninje
- Data Warehousing IntroductionUploaded bysuperstar538
- Presentation 1Uploaded byBhanu Partap Sharma
- segmentationUploaded bysanjaygaur123
- Clustering of Vehicle TrajectoriesUploaded byJack Yuan
- Importance–Performance Analysis by Fuzzy C-Means AlgorithmUploaded byAhmad Rifa'i
- Disease Mapping and Risk Asseement for Public HealthUploaded byBayuSaputra
- 01.04 Clustering ExamplesUploaded byAnurag GV
- Case_study_Uploaded byergarza88
- Zhang Money LaunderingUploaded byluislupe

- Cache CoherenceUploaded bymotassm
- MdxUploaded byAmit Singh
- A Survey of Hardware Solutions for Maintenance of Cache Coherence in Shared Memory MultiprocessorsUploaded byAmit Singh
- Towards a Data Warehouse Testing FrameworkUploaded byAmit Singh
- Temporal and Spatial LocalityUploaded byAmit Singh
- UntitledUploaded byAmit Singh
- AMCAT Test SyllabusUploaded byHarsha Nihanth

- MotherFirst -- Maternity Mental Health Strategy: Building Capacity in SaskatchewanUploaded bykidSKAN Director
- Memo final edition(upload)Uploaded byYao Yang
- Barriers to Innovation in the Implementation of the Investment StrategyUploaded byMarcelo Antonino
- Islamic Relief - Application_FormUploaded byKhursheed Ahmad
- Services PowerUploaded byEd Casin
- CLMS Student Started GuideUploaded byPablo Riquelme González
- Hasibul Islam KhanUploaded byAurel Khan
- Cost of Poor MaintenanceUploaded bySaulo Cabrera
- MBIST GuideUploaded byMajid Rabbani
- Independent Project MemoUploaded bymsplonsk
- Protocol for Designing and Conducting Potato Field Experiments for Modeling PurposesUploaded bycip-library
- Block 1 MS 52 Unit 1Uploaded bydarwante
- GATE-17 Online Test Series Mechanical Engineering - ACE Engineering AcademyUploaded bykkk
- Minum IndexUploaded byBoombasticShaggy
- Teaching With RUploaded byFreeman Jackson
- Notes 4 Sensory AcuityUploaded byR Selva Raja
- PSCafePOS GuideUploaded byAlvin T. Enguillo
- Elastohydrodynamic Lubrication FundamentalsUploaded byGwePark
- information-communication-technologies-for-enhanced-education-and-learning-advanced-appli.pdfUploaded byEduardo Panadero Cuartero
- Manual EngUploaded byCirilo Loreto Torres
- John Carroll University Magazine Fall 2010Uploaded byjohncarrolluniversity
- CVD Outcomes Web1Uploaded byRani Oktaviani Sidauruk
- Concept Note C3D+Uploaded bydanielitom
- using gender to undo gender.docUploaded byJuliana Lye Batista
- lenz lawUploaded byarini_annora
- Technical Visit - Vajont DamUploaded byRazvan
- NCSUploaded byprabhakaran
- The Ideology and Logic of Scientism. b y. David James HardingUploaded byWill Eede
- Survey on Virtual Machine Placement Techniques InUploaded byAnonymous roqsSNZ
- aim-2008-16-4-10.pdfUploaded byThalia Karampasi