3/14/24, 7:16 AM SAP HANA PAL – K-Means Algorithm or How to do Cust...

- SAP Community

In this post I’m going to focus on how to use the K-Means clustering algorithm included in PAL because it’s one of the most
popular and most commonly used in data-mining. But before we jump into the code, let’s talk about how the algorithm
works.

According to Wikipedia, “clustering is the task of grouping a set of objects in a way that objects in the same group (called
cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters)”. In other words,
grouping your data into multiple clusters. The most common use case for a clustering algorithm is customer segmentation,
meaning you use a clustering algorithm to divide your customer database in multiple groups (or clusters) based on how
similar they are or how similar they behave, e.g., age, gender, interests, spending habits and so on.

The K-Means algorithm works in a very simple way (for me that I don’t have to code it in C++ J). The first step is to plot all
the database objects into space where each attribute is a dimension. So if we use a two attributes data set the resulting
chart would look something like this:

After all the objects are plotted, the algorithm calculates the distance between them and the ones that are close to each
other are grouped in the same cluster. So if we go back to our previous example we can create 4 different clusters:

https://community.sap.com/t5/technology-blogs-by-members/sap-hana-pal-k-means-algorithm-or-how -to-do-customer-segmentation-for-the/ba-p/12976696/page/2 2/39

3/14/24, 7:16 AM SAP HANA PAL – K-Means Algorithm or How to do Cust... - SAP Community

Each cluster is associated with a centroid and each point is assigned to the cluster with the closest centroid. The centroid is
the mean of the points in the cluster. The closeness can be measured using:

Manhattan Distance
Euclidean Distance (most commonly used)
Minkowski Distance

Every time a point is assigned to a cluster the centroid is recalculated. This is repeated in multiple iterations until centroids
don’t change anymore (meaning all points have been assigned to a corresponding cluster) or until relatively few points
change clusters. Usually most of the centroid movement happens in the first iterations.

One of the main drawbacks of the K-Means Algorithm is that you need to specify the number of Ks (or clusters) upfront as
an input parameter. Knowing this value is usually very hard, that is why it is important to run quality measurement
functions to check the quality of your clustering. Later in this post we will talk about this.

I came across a very interesting paper that talks about segmentation in the telecommunication industry, so I thought it
would be a very nice use case to demo the K-Means algorithm in HANA (if you are interested in this topic, I very much
recommend reading this paper). These are the steps I followed:

https://community.sap.com/t5/technology-blogs-by-members/sap-hana-pal-k-means-algorithm-or-how -to-do-customer-segmentation-for-the/ba-p/12976696/page/2 3/39

3/14/24, 7:16 AM SAP HANA PAL – K-Means Algorithm or How to do Cust... - SAP Community

/* Table Type that will be used as the input parameter

that will contain the data that I would like to cluster */

DROP TYPE PAL_KMEANS_DATA_TELCO;

CREATE TYPE PAL_KMEANS_DATA_TELCO AS TABLE(

"ID" INT,

"AVG_CALL_DURATION" DOUBLE,

"AVG_NUMBER_CALLS_RCV_DAY" DOUBLE,

"AVG_NUMBER_CALLS_ORI_DAY" DOUBLE,

"DAY_TIME_CALLS" DOUBLE,

"WEEK_DAY_CALLS" DOUBLE,

"CALLS_TO_MOBILE" DOUBLE,

"SMS_RCV_DAY" DOUBLE,

"SMS_ORI_DAY" DOUBLE,

https://community.sap.com/t5/technology-blogs-by-members/sap-hana-pal-k-means-algorithm-or-how -to-do-customer-segmentation-for-the/ba-p/12976696/page/2 7/39