Affinity propagation

In statistics and data mining, affinity propagation (AP) is a clustering algorithm based on the concept of
"message passing" between data points.[1] Unlike clustering algorithms such as k-means or k-medoids,
affinity propagation does not require the number of clusters to be determined or estimated before running
the algorithm. Similar to k-medoids, affinity propagation finds "exemplars," members of the input set that
are representative of clusters.[1]

Algorithm
Let x1 through xn be a set of data points, with no assumptions made about their internal structure, and let s
be a function that quantifies the similarity between any two points, such that s(i, j) > s(i, k) iff xi is more
similar to xj than to xk. For this example, the negative squared distance of two data points was used i.e. for
points xi and xk, [1]

The diagonal of s (i.e. ) is particularly important, as it represents the instance preference, meaning
how likely a particular instance is to become an exemplar. When it is set to the same value for all inputs, it
controls how many classes the algorithm produces. A value close to the minimum possible similarity
produces fewer classes, while a value close to or larger than the maximum possible similarity produces
many classes. It is typically initialized to the median similarity of all pairs of inputs.

The algorithm proceeds by alternating between two message-passing steps, which update two matrices:[1]

The "responsibility" matrix R has values r(i, k) that quantify how well-suited xk is to serve as
the exemplar for xi, relative to other candidate exemplars for xi.
The "availability" matrix A contains values a(i, k) that represent how "appropriate" it would
be for xi to pick xk as its exemplar, taking into account other points' preference for xk as an
exemplar.

Both matrices are initialized to all zeroes, and can be viewed as log-probability tables. The algorithm then
performs the following updates iteratively:

First, responsibility updates are sent around:

Then, availability is updated per

for and

.
Iterations are performed until either the cluster boundaries remain unchanged over a number of iterations, or
some predetermined number (of iterations) is reached. The exemplars are extracted from the final matrices
as those whose 'responsibility + availability' for themselves is positive (i.e. ).

Applications
The inventors of affinity propagation showed it is better for certain computer vision and computational
biology tasks, e.g. clustering of pictures of human faces and identifying regulated transcripts, than k-
means,[1] even when k-means was allowed many random restarts and initialized using PCA.[2] A study
comparing affinity propagation and Markov clustering on protein interaction graph partitioning found
Markov clustering to work better for that problem.[3] A semi-supervised variant has been proposed for text
mining applications.[4] Another recent application was in economics, when the affinity propagation was
used to find some temporal patterns in the output multipliers of the US economy between 1997 and
2017.[5]

Software
A Java implementation is included in the ELKI data mining framework.
A Julia implementation of affinity propagation is contained in Julia Statistics's Clustering.jl
package.
A Python version is part of the scikit-learn library.
An R implementation is available in the "apcluster" package.

References
1. Brendan J. Frey; Delbert Dueck (2007). "Clustering by passing messages between data
points". Science. 315 (5814): 972–976. CiteSeerX 10.1.1.121.3145 (https://citeseerx.ist.psu.
edu/viewdoc/summary?doi=10.1.1.121.3145). doi:10.1126/science.1136800 (https://doi.org/
10.1126%2Fscience.1136800). PMID 17218491 (https://pubmed.ncbi.nlm.nih.gov/1721849
1). S2CID 6502291 (https://api.semanticscholar.org/CorpusID:6502291).
2. Delbert Dueck; Brendan J. Frey (2007). Non-metric affinity propagation for unsupervised
image categorization. Int'l Conf. on Computer Vision. doi:10.1109/ICCV.2007.4408853 (http
s://doi.org/10.1109%2FICCV.2007.4408853).
3. James Vlasblom; Shoshana Wodak (2009). "Markov clustering versus affinity propagation
for the partitioning of protein interaction graphs" (https://www.ncbi.nlm.nih.gov/pmc/articles/P
MC2682798). BMC Bioinformatics. 10 (1): 99. doi:10.1186/1471-2105-10-99 (https://doi.org/
10.1186%2F1471-2105-10-99). PMC 2682798 (https://www.ncbi.nlm.nih.gov/pmc/articles/P
MC2682798). PMID 19331680 (https://pubmed.ncbi.nlm.nih.gov/19331680).
4. Renchu Guan; Xiaohu Shi; Maurizio Marchese; Chen Yang; Yanchun Liang (2011). "Text
Clustering with Seeds Affinity Propagation". IEEE Transactions on Knowledge & Data
Engineering. 23 (4): 627–637. doi:10.1109/tkde.2010.144 (https://doi.org/10.1109%2Ftkde.2
010.144). hdl:11572/89884 (https://hdl.handle.net/11572%2F89884). S2CID 14053903 (http
s://api.semanticscholar.org/CorpusID:14053903).
5. Almeida, Lucas Milanez de Lima; Balanco, Paulo Antonio de Freitas (2020-06-01).
"Application of multivariate analysis as complementary instrument in studies about structural
changes: An example of the multipliers in the US economy" (http://www.sciencedirect.com/s
cience/article/pii/S0954349X17301406). Structural Change and Economic Dynamics. 53:
189–207. doi:10.1016/j.strueco.2020.02.006 (https://doi.org/10.1016%2Fj.strueco.2020.02.0
 06). ISSN 0954-349X (https://www.worldcat.org/issn/0954-349X). S2CID 216406772 (https://
api.semanticscholar.org/CorpusID:216406772).

Retrieved from "https://en.wikipedia.org/w/index.php?title=Affinity_propagation&oldid=1131322116"