K-MEANS-DIRECTIONS ALGORITHM FOR

SEMISUPERVISED ANALYSIS ON THE SPHERE

Last Update: 23rd October 2023

Yeily P. Guzman-Castro
University of Puerto Rico
Mayagüez Campus
Table of Contents Theoretical context Target function Minimization of the Objective function

The objective of this project is to cluster semi-supervised data on a sphere by

minimizing the distance equation between centroids and observations.

2
Table of Contents Theoretical context Target function Minimization of the Objective function

TABLE OF CONTENTS

1. Theoretical context

2. Target function

3. Minimization of the Objective function

3
THEORETICAL CONTEXT
Table of Contents Theoretical context Target function Minimization of the Objective function

SEMI-SUPERVISED DATA

It is a type of data that contains a mixture of labeled and unlabeled examples. In the
context of machine learning and data analysis, data points are considered ”labeled”
when they have associated ground truth or target values, while ”unlabeled” data
points do not have these target values assigned. This is a common scenario in various
real-world applications.

5
Table of Contents Theoretical context Target function Minimization of the Objective function

K-MEANS ALGORITHM

K-means is a popular clustering algorithm used in machine learning and data analysis.
It is an unsupervised learning algorithm that aims is to minimize the sum of squared
distances between data points and their assigned cluster centroids.

6
TARGET FUNCTION
Table of Contents Theoretical context Target function Minimization of the Objective function

Our objective function is a variation of that of Maitra and Ramler[3] for data on the
sphere and Almodovar-Rivera[1] for semi-supervised data.

8
Table of Contents Theoretical context Target function Minimization of the Objective function

FUNCTION APPROACH

Let O = {X1• , X2• , . . . , Xn•• , X1◦ , X2◦ , . . . , Xn◦◦ } be a set of n = n • + n ◦ p-dimensional

observations coming from S p = {X ∈ Rp |X 0 X = 1}. We assume to know the K • > 0
classes of O• = {X1◦ , X2◦ , . . . , Xn◦◦ } of the K possible classes. Let

 0 if Xi• ∈
/ Ck•

•
ζik =
1 if Xi• ∈ Ck•


with k = 1, 2, . . . , K • and

9
Table of Contents Theoretical context Target function Minimization of the Objective function

 0 if Xj◦ ∈
/ Ck◦

◦
ζjk =
1 if Xj◦ ∈ Ck◦


with k = 1, 2, . . . , K . Our goal is to find the class for each unsupervised observation
for which

K X
n• • K X
n ◦

(1 − Xi µk )ζik + (1 − Xj◦0 µk )ζjk

X X
•0 • ◦
(1)
k=1 i=1 k=1 j=1

is minimized.

10
Table of Contents Theoretical context Target function Minimization of the Objective function

FUNCTION REWRITE

In computational and practical terms, it is more convenient to rewrite the supervised

part of the function in terms of the
Paverage of its elements.
multiplying and dividing 1 by the ni=1 ζik
•
• term, we obtain

K • K X
n ◦

nk• (1 Xi•0 µk )ζik (1 − Xj◦0 µk )ζjk

X X
• ◦
− + (2)
k=1 k=1 j=1
.

11
MINIMIZATION OF THE OBJECTIVE FUNCTION
Table of Contents Theoretical context Target function Minimization of the Objective function

CLASS ONLY WITH NON-SUPERVISED OBSERVATIONS

To minimize 2, we must consider different cases. Let us take the case in which there is
a group composed entirely of unsupervised elements, in such a case our equation
results in

K X
n ◦

(1 − Xj◦0 µk )ζjk
X
◦
(3)
k=1 j=1

for K • + ≤ k ≤ K .

13
Table of Contents Theoretical context Target function Minimization of the Objective function

For a certain K • + ≤ k ≤ K , taking any other vector Z ∈ S p and using the

cauchy-Schwarz inequality and inspired by Dhillon [2] , we have that

n ◦

(1 − Xj◦0 Z )ζjk = nk◦ − |kζ1k X1◦ + ζ2k X2◦ + . . . + ζn◦◦ k Xn◦◦ kkZ kCos(θ(Xj , Z ))|
X
◦ ◦ ◦

j=1
q
≥ nk◦ ◦ X ◦ + ζ ◦ X ◦ + . . . + ζ ◦ X ◦ k2 kZ k2 Cos(θ(X , Z ))
− kζ1k 1 2k 2 n◦ k n◦ j
n ◦

(1 − Xj◦0 µk )ζjk
X
◦
=
j=1

14
Table of Contents Theoretical context Target function Minimization of the Objective function

Finding that

n◦
P 
j=1 ζjk Xj
1 ◦ ◦
nk◦
µk =
n◦
P 
k n1◦ j=1 ζjk Xj
◦ ◦ k
k

minimizes to 3.

15
Table of Contents Theoretical context Target function Minimization of the Objective function

MIXED CASE

Let us now analyze the general case, in which we have classes consisting of labeled
and unlabeled elements. In such a case, our equation is still 2. Analogous to the
previous case, we see that

n ◦

nk• (1 Xk•0 ) (1 − Xj◦0 Z )ζjk

X
◦
− +
j=1
n◦

nk• nk◦ |knk• Xk•0 Xj kkZ kCos(θ(Xj , Z ))|

X
◦ ◦
= + − + ζjk
j=1
v
u n◦
≥ nk + nk − knk Xk + ◦ X ◦ k2 kZ k2 Cos(θ(X , Z ))
X
◦ •
u
• •0 ζjk j j
t
j=1

16
Table of Contents Theoretical context Target function Minimization of the Objective function

FRAME TITLE

n ◦

nk• (1 Xi•0 µk )ζik (1 − Xj◦0 µk )ζjk

X
• ◦
= − +
j=1

Finding that

nk• Xk• +nk◦ Xk◦

nk• +nk◦
µk =
nk• Xk• +nk◦ Xk◦
nk• +nk◦

minimizes to 2.

17
Table of Contents Theoretical context Target function Minimization of the Objective function

REFERENCES

Almodóvar-Rivera, Israel A. (2017). Some contributions to k − means clustering

problems. doctoral thesis, Iowa State University.
Dhillon Inderjit S. & Modha Dharmendra S. (2001). Concept Decompositions for
Large Sparse Text Data Using Clustering. Machine Learning, 42, 143–175.
Maitra, R., & Ramler, I. P. (2010). A k-mean-directions algorithm for fast clustering
of data on the sphere. Journal of Computational and Graphical Statistics, 19(2),
377-396.

18