Aman19csu019 ML Lab Expierment 1,2,3,4,5,7

Machine Learning
Lab Manual
Department of Computer Science and Engineering
The NorthCap University, Gurugram
ii
Machine
Learning Lab Manual (CSL 313)
2022
Machine Learning Lab Manual

CSL 313
iii
Machine
2022
NorthCap University, Gurugram- 122001, India
Session 2022
Published by:
School of Engineering and Technology
Department of Computer Science & Engineering
The NorthCap University Gurugram

iv
Machine
2022
• Laboratory Manual is for Internal Circulation only
© Copyright Reserved
No part of this Practical Record Book may be
reproduced, used, stored without prior permission of The NorthCap University
Copying or facilitating copying of lab work comes under cheating and is considered as use
of unfair means. Students indulging in copying or facilitating copying shall be awarded zero
marks for that particular experiment. Frequent cases of copying may lead to disciplinary
action. Attendance in lab classes is mandatory.
Labs are open up to 7 PM upon request. Students are encouraged to make full use of labs
beyond normal lab hours.
PREFACE
Machine Learning Lab Manual is designed to meet the course and program requirements of
NCU curriculum for B.Tech III year students of CSE branch. The concept of the lab work is
to give brief practical experience for basic lab skills to students. It provides the space and
scope for self-study so that students can come up with new and creative ideas.
The Lab manual is written on the basis of “teach yourself pattern” and expected that
students who come with proper preparation should be able to perform the experiments
without any difficulty. Brief introduction to each experiment with information about self-
study material is provided. The pre-requisite is having a basic working knowledge of
Python. The laboratory exercises will include familiarization with unsupervised ML like
Clustering and Association rule mining, introduction to Reinforcement Learning; Feature
Selection and Dimensionality Reduction. Students would learn the algorithms pertaining to
these and implement the same using a high-level language, i.e. Python. Students are
v
Machine
2022
expected to come thoroughly prepared for the lab. General disciplines, safety guidelines
and report writing are also discussed.
The lab manual is a part of curriculum for the The NorthCap University, Gurugram.
Teacher’s copy of the experimental results and answer for the questions are available as
sample guidelines.
We hope that lab manual would be useful to students of CSE, IT, ECE and BSc branches and
author requests the readers to kindly forward their suggestions / constructive criticism for
further improvement of the work book.
Author expresses deep gratitude to Members, Governing Body-NCU for encouragement and
motivation.
Authors
The NorthCap University
Gurugram, India
CONTENTS
S.N. Details Page No.
Syllabus 6
1 Introduction 9
2 Lab Requirement 10
3 General Instructions 11
4 List of Experiments 13
vi
Machine
2022
5 List of Flip Assignment 15
6 List of Projects 16
7 Rubrics 17
8 Annexure 1 (Format of Lab Report) 18
SYLLABUS
1. Department: Department of Computer Science and Engineering
2. Course Name: Machine Learning 3. Course Code 4. L-T-P 5. Credits

CSL313 2-0-4 4
6. Type of Course j
(Check one): Programme Core Programme Elective  Open Elective
d
7. Pre-requisite(s), if any: Introduction to AI and ML
8. Frequency of offering (check one): Odd Even  Either semester Every semester
9. Brief Syllabus:
Overview to machine learning and pre-processing concepts, Model Selection, Model Selection, XGBoost. Feature
Selection- Filter and Wrapper, Dimensionality Reduction, Principal Component Analysis PCA, Linear Discriminant
Analysis LDA, Kernel PCA, Introduction to Self-Organizing Maps (SOM), Building a Self-Organizing Map. Overview
of clustering in machine learning, Different categories of clustering algorithms, similarity/distance measures, K
Means algorithm, Hierarchical, DBSCan, Fuzzy C-Means , Agglomerative clustering algorithm, Expectation
maximization (EM) for soft clustering. Semi-supervised learning with EM using labeled and unlabled data.,
Evaluation methods, A case study with clustering implementation, Eclat, Recommendation System , User Based
Recommendation System, Item Based Recommendation System, Matrix-Factorization Recommendation System
Total lecture, Tutorial and Practical Hours for this course (Take 15 teaching weeks per semester): 90 hours
The class size is maximum 30 learners
Practice
Lectures: 30 hours
Tutorials : 0 hours Lab Work: 60 hours
10. Course Outcomes (COs)
On successful completion of this course students will be able to:
vii
Machine
2022
Understand the difference between supervised and unsupervised approaches and design the model
CO 1
with no training data.
CO 2 Implement the methods to find frequent Patterns and associations in the patterns.
CO 3 Implement Recommendation System with real world data .
CO 4 Implement dimensionality reduction techniques to improve the efficiency of models
Implement the real-world problems with model selection and optimized feature selection for further
CO 5
processing of the data.
11. UNIT WISE DETAILS No. of Units: 5
Unit Number: 1 Title: Clustering No. of hours: 4

Content Summary:
A quick recap to introduction to machine learning, Overview of clustering in machine learning, Different categories
of clustering algorithms, similarity/distance measures, K Means algorithm, Hierarchical Clustering, Advantages,
limitations and comparison.
Unit Number: 2 Title: Association Rule Mining No. of hours:6
Content Summary:
Basic concepts of frequent pattern mining, Frequent Itemset mining methods, Apriori algorithm, Eclat algorithm,
Vertical Data format method, pattern evaluation methods, association analysis to correlation analysis.
Unit Number: 3 Title: Recommendation System No. of hours:6
Content Summary: Introduction to Recommendation System, User Based Recommendation System, Item Based
Recommendation System, Matrix-Factorization Recommendation System.
Unit Number: 4 Title: Dimensionality Reduction No. of hours:6
Content Summary:
Introduction to dimensionality reduction, Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA),
Kernel PCA.
Unit Number: 5 Title: Model Selection and Boosting No. of hours:8
Content Summary: Pre-processing concepts, Model Selection, Model Selection, XGBoost. Feature Selection-
Filter and Wrapper, Introduction to Self-Organizing Maps (SOM), Building a Self-Organizing Map
viii
Machine
2022
12. Brief Description of Self-learning components by students (through books/resource material etc.):
Data preprocessing techniques
13. Course/Reference Books:
1. Machine Learning using Python Paperback – 1 January 2019, by U Dinesh Kumar, Manaranjan Pradhan, Wiley,
https://www.amazon.in/Machine-Learning-Python-Manaranjan-Pradhan/dp/8126579900.
2. Matthew Kirk, “Thoughtful Machine Learning: A Test-Driven Approach” , First Edition, O'Reilly Publications, 2014.
3. Tom Mitchell,”Machine Learning”, McGraw Hill, 1997.
Reference Websites: (nptel, swayam, coursera, edx, udemy, lms, official documentation weblink)
 https://www.coursera.org/learn/practical-rl#syllabus
 https://nptel.ac.in/courses/106/106/106106139/
 https://www.coursera.org/learn/machine-learning
Project (To be done as individual/in group): Individual
1. INTRODUCTION
That ‘learning is a continuous process’ cannot be over emphasized. The theoretical

knowledge gained during lecture sessions need to be strengthened through practical
experimentation. Thus practical makes an integral part of a learning process.
OBJECTIVE:
The purpose of conducting experiments can be stated as follows:
 To familiarize the students with the basic concepts of Machine Learning like
supervised, unsupervised and reinforcement learning.
 The lab sessions will be based on exploring the concepts discussed in class.
 Learning and understanding Clustering.
ix
Machine
2022
 Learning and understanding Association Rule Mining algorithms.

 Learning and understanding Recommendation System.
 Learning and understanding Feature selection and Dimensionality Reduction.
 Hands on experience
2. LAB REQUIREMENTS
S.No. Requirements Details

1 Software Python 3.
Requirements
2 Operating System Windows(64-bit), Linux
3 Hardware 8 GB RAM (Recommended)
Requirements 2.60 GHz (Recommended)
4 Required Bandwidth NA
x
Machine
2022
3. GENERAL INSTRUCTIONS
3.1 General discipline in the lab
 Students must turn up in time and contact concerned faculty for the experiment
they are supposed to perform.
 Students will not be allowed to enter late in the lab.
 Students will not leave the class till the period is over.
 Students should come prepared for their experiment.
 Experimental results should be entered in the lab report format and
certified/signed by concerned faculty/ lab Instructor.
 Students must get the connection of the hardware setup verified before
switching on the power supply.
xi
Machine
2022
 Students should maintain silence while performing the experiments. If any

necessity arises for discussion amongst them, they should discuss with a very
low pitch without disturbing the adjacent groups.
 Violating the above code of conduct may attract disciplinary action.
 Damaging lab equipment or removing any component from the lab may invite
penalties and strict disciplinary action.
3.2 Attendance
 Attendance in the lab class is compulsory.

 Students should not attend a different lab group/section other than the one
assigned at the beginning of the session.
 On account of illness or some family problems, if a student misses his/her lab
classes, he/she may be assigned a different group to make up the losses in
consultation with the concerned faculty / lab instructor. Or he/she may work
in the lab during spare/extra hours to complete the experiment. No attendance
will be granted for such case.
3.3 Preparation and Performance
 Students should come to the lab thoroughly prepared on the experiments they
are assigned to perform on that day. Brief introduction to each experiment
with information about self study reference is provided on LMS.
 Students must bring the lab report during each practical class with written
records of the last experiments performed complete in all respect.
 Each student is required to write a complete report of the experiment he has
performed and bring to lab class for evaluation in the next working lab.
Sufficient space in work book is provided for independent writing of theory,
observation, calculation and conclusion.
 Students should follow the Zero tolerance policy for copying / plagiarism. Zero
marks will be awarded if found copied. If caught further, it will lead to
disciplinary action.
 Refer Annexure 1 for Lab Report Format
xii
Machine
2022
4. LIST OF EXPERIMENTS
Sr. Title of the Experiment Software Unit CO
No. used Covered Covered
1. Perform K-means clustering Algorithm. Write the python Python 1 CO1
code to find Intercluster distance and IntraCluster (Jupyter)
Distance.
2. Perform K means Clustering on given dataset. Identify Python 1 CO1
number of clusters using elbow method. (Jupyter)
3. Perform hierarchical clustering on given dataset. Identify Python 1 CO1
number of clusters using dendograms. (Jupyter)
4. Compare K-means and Hierarchical clustering for any Python 1 CO1
xiii
Machine
2022
one dataset. Calculate the Silhouette Score. (Jupyter)

5. Apply Apriori algorithm to identify frequent itemsets from Python 2 CO2
the bakery dataset. (Jupyter)
6. Apply Eclat algorithm to identify frequent itemsets from Python 2 CO2
the bakery dataset. (Jupyter)
7. Apply FP grow algorithm to identify frequent itemsets Python 2 CO2
from the bakery dataset. (Jupyter)
8. To apply User Based Recommendation System Python 3 CO3
(Jupyter)
9. Python 3 CO3
To apply Item Based Recommendation System (Jupyter)
10. Python 3 CO3
To apply Matrix-Factorization Recommendation System. (Jupyter)
11. Python 4 CO4

Reduce dimensionality of Iris dataset using PCA (Jupyter)
12.
Reduce dimensionality of Iris dataset using LDA
Python 4 CO4
(Jupyter)
13. To apply five Filter feature selection techniques. Python 5 CO5
(Jupyter)
14. To apply Recursive Feature Elimination. Python 5 CO5
(Jupyter)
15. To apply XGBoost on two datasets. Python 5 CO5
(Jupyter)
16. Construct Self Organizing Maps on Iris dataset. Python 5 CO5
(Jupyter)
18. Build a ML model from scratch using and comparing Python 1,2,3,4 CO1,CO2,
supervised, unsupervised and ensemble ML (Jupyter) CO3,CO4
approaches.
19. Build a ML model from scratch applying and comparing Python 1,2,3,4 CO1,CO2,
filter, wrapper, embedded and hybrid feature selection (Jupyter) CO3,CO4
approaches
xiv
Machine
2022
5. LIST OF FLIP EXPERIMENTS
5.1 Project – supervised and unsupervised learning based model on market basket
analysis
5.2 Competition on Kaggle
6. LIST OF PROJECTS
1. Titanic Challenge: The sinking of the Titanic is one of the most infamous
shipwrecks in history. On April 15, 1912, during her maiden voyage, the widely
considered “unsinkable” RMS Titanic sank after colliding with an iceberg.
Unfortunately, there weren’t enough lifeboats for everyone onboard, resulting in
the death of 1502 out of 2224 passengers and crew. While there was some
element of luck involved in surviving, it seems some groups of people were more
likely to survive than others. In this project, the students need to build a predictive
xv
Machine
2022
model that answers the question: “what sorts of people were more likely to
survive?” using passenger data (ie name, age, gender, socio-economic class, etc).
2. House Price Prediction Using Advanced Regression Techniques: Ask a home buyer
to describe their dream house, and they probably won't begin with the height of
the basement ceiling or the proximity to an east-west railroad. But Kaggle’s
advanced house price prediction dataset proves that much more influences price
negotiations than the number of bedrooms or a white-picket fence. With 79
explanatory variables describing (almost) every aspect of residential homes in
Ames, Iowa, this dataset can be used to predict the final price of each home.
3. Mechanism of Action (MoA) Prediction: Mechanism of action means the
biochemical interactions through which a drug generates its pharmacological
effect. If we know a disease affects some particular receptor or downstream set of
cell activity, we can develop drugs faster if we can predict how cells and genes
affect various receptor sites. Using a dataset that combines gene expression and
cell viability data in addition to the MoA annotations of more than 5,000 drugs. In
this, each drug was tested under two dose (cp_dose) and three times (cp_time). So,
six samples basically correspond to one drug. We need to train a model that
classifies drugs based on their biological activity. This problem is a multi-label
classification, which means we have multiple targets (not multiple classes). In this
project, perform explanatory data analysis and then train a model using deep
neural networks with Keras.
7. RUBRICS
Marks Distribution
Continuous Evaluation (15 Marks) Project Evaluations (80 Marks)
Each experiment shall be evaluated for 10 Both the projects shall be evaluated for
marks and at the end of the semester 30 marks each and at the end of the
proportional marks shall be awarded out semester viva will be conducted related
of total 15. to the projects as well as concepts
Following is the breakup of 10 marks for learned in labs and this component
each carries 20 marks.
xvi
Machine
2022
4 Marks: Observation & conduct of

experiment. Teacher may ask questions
about experiment.
3 Marks: For report writing
3 Marks: For the 15 minutes quiz to be
conducted in every lab.
Annexure 1
Machine Learning
(CSL 313)
Lab Practical Report

xvii
Machine
2022
Faculty name: Dr. Vidhi Khanduja Student name: Aman Parkash
Roll No.: 19CSU019
Semester: 6th
Group: DS-E-9

NorthCap University, Gurugram- 122001, India
Session 2022
xviii
Machine
2022
INDEX
S.No Experiment Page Date of Date of Marks CO Sign
No. Experiment Submission Covered
1 Perform K-means
clustering Algorithm. Write
the python code to find
Intercluster distance and
IntraCluster Distance.
2 Perform K means
Clustering on given
dataset. Identify
number of clusters
using elbow method.
3 Perform hierarchical
clustering on given
dataset. Identify number
of clusters using
dendograms.
4 Compare K-means and

Hierarchical clustering for
any one dataset.
Calculate the Silhouette
Score.
5 Apply Apriori algorithm to

identify frequent itemsets
from the bakery dataset.
6 Apply Eclat algorithm to

identify frequent itemsets
from the bakery dataset.
7 Apply FP grow algorithm

to identify frequent
itemsets from the bakery
dataset.
xix
Machine
2022
EXPERIMENT NO. 1
Student Name and Roll Number: Aman Parkash and 19CSU019
Semester /Section: 6th / DS-E-9
Link to Code:
Date:
Faculty Signature:
Grade:
xx
Machine
2022
Objective(s):
 Understand what is unsupervised learning.

 Study about clustering and different clustering approaches.
 Implement k-means clustering for solving a real-world problem.
Outcome:
Students would be familiarized with unsupervised learning and clustering in particular.
Problem Statement:
Perform K-means clustering Algorithm. Write the python code to find Inter-cluster distance and Intra-Cluster
Distance.
Background Study: K-means clustering is a method of vector quantization, originally from signal
processing, that aims to partition n observations into k clusters in which each observation belongs
to the cluster with the nearest mean (cluster centers or cluster centroid), serving as a prototype of
the cluster.
Question Bank:
1. What is the difference between supervised and unsupervised learning?
2. What is clustering?
3. What are the different types of clustering approaches?
4. How to find the optimal value of k in k-means clustering?
Student Work Area

Algorithm/Flowchart/Code/Sample Outputs
xxi
Machine
2022
import seaborn as sns
import numpy as np
import matplotlib.pyplot as plt
import math
%matplotlib inline
from sklearn.datasets import make_blobs
# Create Data
data = make_blobs(n_samples=200, n_features=2,
centers=4, cluster_std=1.8,random_state=10,return_centers = True)
# n_samples
# If we pass int, it is the total number of points equally divided among clusters.
# and If we pass array, each element of the sequence indicates the number of samples per cluster.
# n_features
# The number of features for each sample.
# centers
# The number of centers to generate, or the fixed center locations.
# If n_samples is an int and centers is None, 3 centers are generated.

xxii
Machine
2022
# If n_samples is array-like, centers must be either None or an array of length equal to the length
of n_samples.
# cluster_std
# The standard deviation of the clusters
# shuffle bool, default=True
# used to shuffle the samples.
# random_state********************
# return_centers
# Return the center od each cluster
plt.scatter(data[0][:,0],data[0][:,1],c=data[1],cmap='rainbow')
xxiii
Machine
2022
kmeans = KMeans(n_clusters=4)
kmeans.fit(data[0])
kmeans.cluster_centers_
kmeans.labels_
f, (ax1, ax2) = plt.subplots(1, 2, sharey=True,figsize=(10,6))
ax1.set_title('K Means')
ax1.scatter(data[0][:,0],data[0][:,1],c=kmeans.labels_,cmap='rainbow')
ax2.set_title("Original")
ax2.scatter(data[0][:,0],data[0][:,1],c=data[1],cmap='plasma')
xxiv
Machine
2022
import numpy as np
cluster1 = []
cluster2 = []
cluster3 = []
cluster4 = []
for i in range(0, 4):
print(f"Inter cluster distance of points from cluster {i} : ")
for j in range(0,200):
if kmeans.labels_[j] == i:
distance = math.dist(kmeans.cluster_centers_[i], data[0][j])
if i==0:
cluster1.append(distance)
xxv
Machine
2022
print(distance)
elif i==1:
print(distance)
elif i==2:
print(distance)
else:
print(distance)
#distance between the centroids of each cluster.
inter_cluster =[]
for i in range(len(kmeans.cluster_centers_)):
print("Distance of cluster", i, "centroid with the rest cluster")

xxvi
Machine
2022
for center in kmeans.cluster_centers_:
distance = math.dist(center, kmeans.cluster_centers_[i])
if(not distance == 0):
inter_cluster.append(distance)
print(distance)
EXPERIMENT NO. 2
Semester /Section:6th / DS-E-9
Link to Code:
xxvii
Machine
2022
Date:
Faculty Signature:
Grade:
Objective(s):
 Understand supervised and unsupervised learning.
 Study about how to find the number of clusters in Kmeans clustering approach
Outcome:
Students would be familiarized with unsupervised learning and K-means clustering in particular.
Problem Statement:
Perform K means Clustering on given dataset. Identify number of clusters using elbow method .
Background Study:
K-means clustering is a method of vector quantization, originally from signal processing, that aims
to partition n observations into k clusters in which each observation belongs to the cluster with the
nearest mean (cluster centers or cluster centroid), serving as a prototype of the cluster.
In cluster analysis, the elbow method is a heuristic used in determining the number of clusters
in a data set. The method consists of plotting the explained variation as a function of the number
of clusters, and picking the elbow of the curve as the number of clusters to use.
Question Bank:
2. What is K-means clustering?

xxviii
Machine
2022
3. What are the different types of clustering approaches?
4. How to find the optimal value of k in k-means clustering?
Student Work Area

from sklearn.cluster import KMeans
#Finding the value of K
wcss = []
for i in range(1,15):
kmeans = KMeans(n_clusters=i)
kmeans.fit_predict(data[0])
wcss.append(kmeans.inertia_)
xxix
Machine
2022
wcss
plt.title("Elbow Method")
plt.plot(range(1,15),wcss)
kmeans = KMeans(n_clusters=4)
kmeans.fit(data[0])
kmeans.cluster_centers_
kmeans.labels_
f, (ax1, ax2) = plt.subplots(1, 2, sharey=True,figsize=(10,6))

xxx
Machine
2022
ax1.set_title('K Means')
ax1.scatter(data[0][:,0],data[0][:,1],c=kmeans.labels_,cmap='rainbow')
ax2.set_title("Original")
ax2.scatter(data[0][:,0],data[0][:,1],c=data[1],cmap='plasma')
xxxi
Machine
2022
EXPERIMENT NO. 3
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):

 Implement both Hierarchical clustering algorithm on any one dataset.
Outcome:
Student will be familiarized with the different types of clustering algorithms.
Problem Statement:
Perform hierarchical clustering on given dataset. Identify number of clusters using dendograms
Background Study:
In data mining and statistics, hierarchical clustering (also called hierarchical cluster
analysis or HCA) is a method of cluster analysis which seeks to build a hierarchy of clusters.
xxxii
Machine
2022
Strategies for hierarchical clustering generally fall into two types:

 Agglomerative: This is a "bottom-up" approach: each observation starts in its own
cluster, and pairs of clusters are merged as one moves up the hierarchy.
 Divisive: This is a "top-down" approach: all observations start in one cluster, and splits
are performed recursively as one moves down the hierarchy.
In general, the merges and splits are determined in a greedy manner. The results of hierarchical
clustering are usually presented in a dendrogram.
Question Bank:
1. What is a dendrogram?
2. What is Hierarchical Agglomerative clustering?
3. What is the linkage criteria used in Hierarchical clustering?
4. What are the different distance metrics that can be used for Hierarchical clustering?
Student Work Area

# Hierarchical Clustering
# Importing the libraries
import numpy as np
%matplotlib inline
import pandas as pd
# Importing the dataset

xxxiii
Machine
2022
dataset = pd.read_csv('Mall_Customers.csv')
dataset
X = dataset.loc[:, ['Age','Annual Income (k$)']].values
plt.figure(figsize=(10,7))
plt.scatter(X[["Age"]],X[["Annual Income (k$)"]],s=100,c='blue')
# Using the dendrogram to find the optimal number of clusters
import scipy.cluster.hierarchy as sch
dendrogram = sch.dendrogram(sch.linkage(X, method = 'ward'))
plt.title('Dendrogram')
plt.xlabel('Customer')
plt.ylabel('Euclidean distances')
xxxiv
Machine
2022
plt.axhline(y=250, color='r',linestyle='-')
plt.show()
# Fitting Hierarchical Clustering to the dataset
from sklearn.cluster import AgglomerativeClustering
cluster = AgglomerativeClustering(n_clusters = 2, affinity = 'euclidean', linkage = 'ward')
cluster=cluster.fit_predict(X)
# Visualising the clusters
plt.scatter(X[cluster == 0, 0], X[cluster == 0, 1], s = 100, c = 'red', label = 'Cluster 1')
plt.scatter(X[cluster == 1, 0], X[cluster == 1, 1], s = 100, c = 'blue', label = 'Cluster 2')
plt.scatter(X[cluster == 2, 0], X[cluster == 2, 1], s = 100, c = 'green', label = 'Cluster3')

xxxv
Machine
2022
plt.scatter(X[cluster == 3, 0], X[cluster == 3, 1], s = 100, c = 'cyan', label = 'Cluster 4')
plt.scatter(X[cluster == 4, 0], X[cluster == 4, 1], s = 100, c = 'magenta', label = 'Cluster5')
plt.title('Clusters of customers')
plt.xlabel('Annual Income (k$)')
plt.ylabel('Spending Score (1-100)')
plt.legend()
plt.show()
xxxvi
Machine
2022
EXPERIMENT NO. 4
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):

 Implement both Hierarchical and K-means clustering algorithm on any one dataset.
 Compare and contrast Hierarchical and K-means clustering algorithm.
Outcome:
Students will be familiarized with clustering algorithms.
Problem Statement:
Compare K-means and Hierarchical clustering for any one dataset. Calculate the Silhouette Score.
Background Study:
xxxvii
Machine
2022
K-means is method of cluster analysis using a pre-specified no. of clusters. It requires advance
knowledge of ‘K’. Hierarchical clustering also known as hierarchical cluster analysis (HCA) is
also a method of cluster analysis which seeks to build a hierarchy of clusters without having
fixed number of clusters. In hierarchical clustering one can stop at any number of clusters and
find appropriate clusters by interpreting the dendrogram.
Question Bank:
1. What are the advantages and disadvantages of K-means vs hierarchical clustering.
2. Compare the time complexities of K-means vs hierarchical clustering.
Student Work Area

import time
import warnings
import numpy as np
import pandas as pd
from sklearn import cluster, datasets

xxxvii
i
Machine
2022
from sklearn.preprocessing import StandardScaler
from itertools import cycle, islice
np.random.seed(0)
n_samples = 1500
noisy_circles = datasets.make_circles(n_samples=n_samples, factor=0.5, noise=0.05)
noisy_moons = datasets.make_moons(n_samples=n_samples, noise=0.05)
blobs = datasets.make_blobs(n_samples=n_samples, random_state=8)
no_structure = np.random.rand(n_samples, 2), None
# Anisotropicly distributed data
random_state = 170
X, y = datasets.make_blobs(n_samples=n_samples, random_state=random_state)
transformation = [[0.6, -0.6], [-0.4, 0.8]]
X_aniso = np.dot(X, transformation)
aniso = (X_aniso, y)
# blobs with varied variances
varied = datasets.make_blobs(
n_samples=n_samples, cluster_std=[1.0, 2.5, 0.5], random_state=random_state
)
xxxix
Machine
2022
# Set up cluster parameters
plt.figure(figsize=(9 * 1.3 + 2, 14.5))
plt.subplots_adjust(
left=0.02, right=0.98, bottom=0.001, top=0.96, wspace=0.05, hspace=0.01
plot_num = 1
default_base = {"n_neighbors": 10, "n_clusters": 3}
datasets = [
(noisy_circles, {"n_clusters": 2}),
(noisy_moons, {"n_clusters": 2}),
(varied, {"n_neighbors": 2}),
(aniso, {"n_neighbors": 2}),
(blobs, {}),
(no_structure, {}),
for i_dataset, (dataset, algo_params) in enumerate(datasets):

xl
Machine
2022
# update parameters with dataset-specific values
params = default_base.copy()
params.update(algo_params)
X, y = dataset
# normalize dataset for easier parameter selection
X = StandardScaler().fit_transform(X)
# ============
# Create cluster objects
# ============
ward = cluster.AgglomerativeClustering(
n_clusters=params["n_clusters"], linkage="ward"
complete = cluster.AgglomerativeClustering(
n_clusters=params["n_clusters"], linkage="complete"
average = cluster.AgglomerativeClustering(
n_clusters=params["n_clusters"], linkage="average"
single = cluster.AgglomerativeClustering(
xli
Machine
2022
n_clusters=params["n_clusters"], linkage="single"
clustering_algorithms = (
("Single Linkage", single),
("Average Linkage", average),
("Complete Linkage", complete),
("Ward Linkage", ward),
for name, algorithm in clustering_algorithms:
t0 = time.time()
# catch warnings related to kneighbors_graph
with warnings.catch_warnings():
warnings.filterwarnings(
"ignore",
message="the number of connected components of the "
+ "connectivity matrix is [0-9]{1,2}"
+ " > 1. Completing it to avoid stopping the tree early.",
category=UserWarning,
)
xlii
Machine
2022
algorithm.fit(X)
t1 = time.time()
if hasattr(algorithm, "labels_"):
y_pred = algorithm.labels_.astype(int)
else:
y_pred = algorithm.predict(X)
plt.subplot(len(datasets), len(clustering_algorithms), plot_num)
if i_dataset == 0:
plt.title(name, size=18)
colors = np.array(
list(
islice(
cycle(
"#377eb8",
"#ff7f00",
"#4daf4a",
"#f781bf",
"#a65628",
xliii
Machine
2022
"#984ea3",
"#999999",
"#e41a1c",
"#dede00",
),
int(max(y_pred) + 1),
plt.scatter(X[:, 0], X[:, 1], s=10, color=colors[y_pred])
plt.xlim(-2.5, 2.5)
plt.ylim(-2.5, 2.5)
plt.xticks(())
plt.yticks(())
plt.text(
0.99,
0.01,
("%.2fs" % (t1 - t0)).lstrip("0"),
transform=plt.gca().transAxes,
size=15,
xliv
Machine
2022
horizontalalignment="right",
plot_num += 1
plt.show()
xlv
Machine
2022
dendrogram = sch.dendrogram(sch.linkage(X, method = 'ward'))
plt.xlabel('data')
plt.show()
dendrogram = sch.dendrogram(sch.linkage(X, method = 'single'))
xlvi
Machine
2022
plt.xlabel('data')
plt.show()
dendrogram = sch.dendrogram(sch.linkage(X, method = 'complete'))
plt.xlabel('data')
plt.show()
xlvii
Machine
2022
dendrogram = sch.dendrogram(sch.linkage(X, method = 'average'))
plt.xlabel('data')
plt.show()
xlviii
Machine
2022
cluster = AgglomerativeClustering(n_clusters = 3, affinity = 'euclidean', linkage = 'ward')
plt.scatter(X[cluster == 1, 0], X[cluster == 1, 1], s = 100, c = 'orange', label = 'Cluster 2')
plt.legend()
plt.show()
xlix
Machine
2022
cluster = AgglomerativeClustering(n_clusters = 3, affinity = 'euclidean', linkage = 'single')
plt.scatter(X[cluster == 1, 0], X[cluster == 1, 1], s = 100, c = 'orange', label = 'Cluster 2')
plt.legend()
plt.show()
l
Machine
2022
import pandas as pd
import numpy as np
import seaborn as sns
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import MinMaxScaler
X = df.iloc[:,:].values
km = KMeans(n_clusters=4)
y_means = km.fit_predict(X)
score = silhouette_score(X, km.labels_,metric='euclidean')

li
Machine
2022
print('Silhoutte Score: %.3f'%score)
EXPERIMENT NO. 5
Student Name and Roll Number: Aman Parkash & 19csu019
Link to Code:
Date:
lii
Machine
2022
Faculty Signature:
Grade:
Objective(s):
 Understand and study Association Rule Mining.

 Study the Apriori property and Apriori Rule in Association Rule Mining.
 Implement the Apriori algorithm in Association Rule Mining.
Outcome:
Students will be familiarized with Association Rule Mining.
Problem Statement:
Apply Apriori algorithm to identify frequent itemsets from the bakery dataset.
Background Study:
Apriori is an algorithm for frequent item set mining and association rule learning over relational
databases. It proceeds by identifying the frequent individual items in the database and extending
them to larger and larger item sets as long as those item sets appear sufficiently often in the
database.
Question Bank:
1. What is the Association Rule Mining?
2. What is the Apriori property?
3. Does Association Rule Mining fall under supervised or unsupervised ML?
4. List the applications of Association Rule Mining.
Student Work Area

liii
Machine
2022
# Importing the libraries
import numpy as np
import pandas as pd
# Data Loading
dataset = pd.read_csv('Market_Basket_Optimisation.csv', header = None)
# Adding all customers into a list of lists
transactions = []
for i in range(0, len(dataset)):
transactions.append([str(dataset.values[i,j]) for j in range(0, 20)])

liv
Machine
2022
# Inspecting elements
transactions[:3]
# Training Apriori on the dataset
# The hyperparameters choosen on this training are:
# min_support = items bought more than 3 times a day * 7 days (week) / 7500 customers =
0.0028
lv
Machine
2022
# min_confidence: at least 20%, min_lift = minimum of 3 (less than that is too low)
# min_length: we want at least 2 items to be associated. No point in having a single item in the
result
from apyori import apriori
rules = apriori(transactions, min_support = 0.003, min_confidence = 0.2, min_lift = 3,

min_length = 2)
# Visualising the results
results = list(rules)
lift = []
association = []
for i in range (0, len(results)):

lvi
Machine
2022
lift.append(results[:len(results)][i][2][0][3])
association.append(list(results[:len(results)][i][0]))
rank = pd.DataFrame([association, lift]).transpose()
rank.columns = ['Association', 'Lift']
# Show top 10 higher lift scores
rank.sort_values('Lift', ascending=False).head(10)
lvii
Machine
2022
EXPERIMENT NO. 6
Semester /Section:6th / DS-E-9
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):

 Study the ECLAT algorithm in Association Rule Mining.
lviii
Machine
2022
 Implement the ECLAT algorithm in Association Rule Mining.
Outcome:
Problem Statement:
Apply ECLAT algorithm to identify frequent itemsets from the bakery dataset.
Background Study:
The ECLAT algorithm stands for Equivalence Class Clustering and bottom-up Lattice Traversal. It
is one of the popular methods of Association Rule mining. It is a more efficient and scalable
version of the Apriori algorithm.
Question Bank:
1. What is the ECLAT algorithm?
2. What are the advantages of ECLAT algorithm over Apriori?
Student Work Area

lix
Machine
2022
EXPERIMENT NO. 7
Semester /Section: DS-E-9
Link to Code:
Date:
lx
Machine
2022
Faculty Signature:
Grade:
Objective(s):

 Study the FP-growth algorithm in Association Rule Mining.
 Implement the FP-growth algorithm in Association Rule Mining.
Outcome:
Problem Statement:
Apply FP-growth algorithm to identify frequent itemsets from the bakery dataset.
Background Study:
FP Growth Algorithm (Frequent pattern growth). FP growth algorithm is an improvement of Apriori

algorithm. FP growth algorithm used for finding frequent itemset in a transaction database without
candidate generation. FP growth represents frequent items in frequent pattern trees or FP-tree.
Question Bank:
1. What is a FP-tree?
2. What is the FP-growth algorithm?
3. List out the advantages and disadvantages of FP-growth algorithm.
4. Compare and contrast Apriori, ECLAT and FP-Growth algorithm.

lxi
Machine
2022
Student Work Area

import pandas as pd
from mlxtend.preprocessing import TransactionEncoder
from mlxtend.frequent_patterns import association_rules
from mlxtend.frequent_patterns import fpgrowth
dataset = [['TFIOS', '5 feet apart', 'Walk to remember'], ['TFIOS', 'Walk to remember'],
['Divergent', 'Harry Potter','Walk to remember'], ['TFIOS', 'Five feet apart'],
['Divergent','Harry Potter']]
te = TransactionEncoder()
te_array = te.fit(dataset).transform(dataset)
df = pd.DataFrame(te_array, columns=te.columns_)
lxii
Machine
2022
frequent_itemsets_fp=fpgrowth(df, min_support=0.02, use_colnames=True)
frequent_itemsets_fp
lxiii
Machine
2022
rules_fp = association_rules(frequent_itemsets_fp, metric="confidence", min_threshold=0.8)

lxiv
Machine
2022
EXPERIMENT NO. 08
Student Name and Roll Number:

lxv
Machine
2022
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 Understand and study Recommendation Learning.

 Study User Based Recommendation System.
Outcome:
Students will be familiarized with User Based Recommendation System.
Problem Statement:
Implement User Based Recommendation System.
Background Study:
Recommender systems are information filtering systems that help deal with the problem of
information overload by filtering and segregating information and creating fragments out of large
amounts of dynamically generated information according to user’s preferences, interests, or
observed behavior about a particular item or items.
User-Based Collaborative Filtering is a technique used to predict the items that a user might like
on the basis of ratings given to that item by the other users who have similar taste with that of the
target user.
Question Bank:
1. What is Recommendation System?

lxvi
Machine
2022
2. Illustrate User Based Recommendation System.
Student Work Area

lxvii
Machine
2022
EXPERIMENT NO. 09
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 Understand and study Recommendation System.

 Introduction to the Item Based Recommendation System.
Outcome:
Students will be familiarized with Item Based Recommendation System.
Problem Statement:
Implement Item Based Recommendation System.
Background Study:
Item-item collaborative filtering, or item-based, or item-to-item, is a form of collaborative filtering for

recommender systems based on the similarity between items calculated using people's ratings of
lxviii
Machine
2022
those items.
Question Bank:
1. What is Recommendation System?
2. Illustrate Item Based Recommendation System.
Student Work Area

lxix
Machine
2022
EXPERIMENT NO. 10
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 Understand and study Recommendation System.

 Introduction to the Matrix-Factorization Recommendation System.
Outcome:
Students will be familiarized with Matrix-Factorization Recommendation System.

lxx
Machine
2022
Problem Statement:
Implement Matrix-Factorization Recommendation System.
Background Study:
Matrix factorization is a way to generate latent features when multiplying two different kinds of
entities. Collaborative filtering is the application of matrix factorization to identify the relationship
between items’ and users’ entities
Question Bank:
1. What is Matrix-Factorization Recommendation System?
Student Work Area

lxxi
Machine
2022
EXPERIMENT NO. 11
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
lxxii
Machine
2022
Objective(s):
 Study Dimensionality Reduction.

 Understand the basic principle behind Principal Component Analysis.
Outcome:
Students will be familiarized with Dimensionality Reduction especially Principal Component

Analysis (PCA).
Problem Statement:
Reduce dimensionality of Iris dataset using Principal Component Analysis.
Background Study:
Principal component analysis is a statistical technique that is used to analyze the interrelationships
among a large number of variables and to explain these variables in terms of a smaller number of
variables, called principal components, with a minimum loss of information.
Question Bank:
1. What is dimensionality reduction?
2. Differentiate between Feature Selection, Feature Engineering and Dimensionality Reduction.
3. What are principal components?
Student Work Area

lxxiii
Machine
2022
EXPERIMENT NO. 12
Semester /Section:
lxxiv
Machine
2022
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 Study Dimensionality Reduction.

 Understand the basic principle behind Linear Discriminant Analysis.
Outcome:
Students will be familiarized with Dimensionality Reduction especially Linear Discriminant Analysis
(LDA).
Problem Statement:
Reduce dimensionality of Iris dataset using Linear Discriminant Analysis.
Background Study:
Linear Discriminant Analysis or Normal Discriminant Analysis or Discriminant Function

Analysis is a dimensionality reduction technique which is commonly used for the supervised
classification problems. It is used for modeling differences in groups i.e., separating two or more
classes. It is used to project the features in higher dimension space into a lower dimension
space.
Question Bank:
1. What is dimensionality reduction?
2. Compare and contrast PCA and LDA for dimensionality reduction.

lxxv
Machine
2022
3. What are the QDA, FDA and RDA extensions to LDA?
4. List the applications of LDA.
Student Work Area

lxxvi
Machine
2022
EXPERIMENT NO. 13
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 To understand the importance of feature selection

 To differentiate between different types of feature selection.
 Build a model using feature selection techniques.
Outcome:
Students will be familiarized with model building using feature selection techniques and
optimization.
Problem Statement:
Write a program to apply any five filter feature selection techniques.
Background Study:
lxxvii
Machine
2022
Feature selection is the process of reducing the number of input variables when developing a
predictive model. It is desirable to reduce the number of input variables to both reduce the
computational cost of modeling and, in some cases, to improve the performance of the model.
Question Bank:
1. Differentiate between Feature Selection and Dimensionality Reduction.
2. What are the advantages of Wrapper methods over filter methods for feature selection?
3. Explain Regularization methods for Feature Selection.
4. What are Embedded feature selection methods.
Student Work Area

lxxvii
i
Machine
2022
EXPERIMENT NO. 14
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 To understand the importance of feature selection

 To differentiate between different types of feature selection.
 To understand the importance of Hybrid Feature Selection Approaches.
 Build a model using feature selection techniques.
lxxix
Machine
2022
Outcome:
Students will be familiarized with model building using feature selection techniques and
optimization.
Problem Statement:
Write a program to perform Recursive Feature Elimination.
Background Study: Recursive Feature Elimination (RFE) is popular feature selection method
because it is easy to configure and use and because it is effective at selecting those features
(columns) in a training dataset that are more or most relevant in predicting the target variable.
There are two important configuration options when using RFE: the choice in the number of
features to select and the choice of the algorithm used to help choose features. Both of these
hyperparameters can be explored, although the performance of the method is not strongly
dependent on these hyperparameters being configured well.
Question Bank:
1. What is Recursive Feature Elimination?
2. What are Hybrid Feature Selection Techniques?
3. Differentiate between Recursive Feature Elimination and Step Backward Feature Selection.
lxxx
Machine
2022
Student Work Area

lxxxi
Machine
2022
EXPERIMENT NO. 15
Semester /Section:
Link to Code:
Date:
Faculty Signature:
lxxxii
Machine
2022
Grade:
Objective(s):
 To study Ensemble ML algorithms.

 To build an efficient model using Ensemble Machine Learning algorithms.
 To understand and implement boosting.
Outcome:
Students will be able to understand how to build a highly efficient ML model (with high
performance), faster.
Problem Statement:
To apply the XGBoost algorithm on two datasets.
Background Study: XGBoost stands for “Extreme Gradient Boosting”. XGBoost is a decision-
tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework.
XGBoost is an optimized distributed gradient boosting library designed to be highly efficient,
flexible and portable. It implements Machine Learning algorithms under the Gradient Boosting
framework. It provides a parallel tree boosting to solve many data science problems in a fast and
accurate way.
Question Bank:
1. What are Ensemble ML algorithms? What is their advantage?
2. Differentiate between bagging and boosting.
3. Differentiate between Boosting, Gradient Boosting and Extreme Gradient Boosting.
4. What are the advantages offered by the XGBoost algorithm that make it superior to most
existing ML algorithms?
lxxxiii
Machine
2022
Student Work Area

EXPERIMENT NO. 16
lxxxiv
Machine
2022
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 Study the basics of dimensionality reduction.

 Understand Self Organizing Maps
 Be able to build self-organizing maps.
Outcome:
Problem Statement:
Construct Self Organizing Maps on Iris dataset.
Background Study: A self-organizing map (SOM) or self-organizing feature map (SOFM) is a

type of artificial neural network (ANN) that is trained using unsupervised learning to produce a low-
dimensional (typically two-dimensional), discretized representation of the input space of the
training samples, called a map, and is therefore a method to do dimensionality reduction.
Question Bank:
1. Why do we need self-organizing maps?
2. What is the difference between feature selection and dimensionality reduction?

lxxxv
Machine
2022
3. What are Artificial neural Networks?
Student Work Area

lxxxvi
Machine
2022
EXPERIMENT NO. 18
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
Objective(s):
 Revise the concepts of supervised and unsupervised learning.

 Use supervised, unsupervised, ensemble ML approaches to solve a real-world problem.
 Build a ML model from scratch and understand the issues involved.
Outcome:
Students will be able to apply the model building concepts learned during ML and use to them
solve a real-world problem.
lxxxvi
i
Machine
2022
Problem Statement:
Build a ML model from scratch using and comparing supervised, unsupervised ML approaches.
Background Study:
In machine learning, algorithms are trained according to the data available and the research
question at hand. But if researchers fail to identify the objective of the machine learning algorithm,
they will not be able to build an accurate model. Ultimately, the researchers need to identify the
problem at hand and which approaches would be more suitable to address the problem.
Question Bank:
2. What is the bias-variance tradeoff in ML?
3. List some unsupervised and supervised ML approaches.
4. In what situations should you use supervised vs unsupervised ML?
Student Work Area

lxxxvi
ii
Machine
2022
EXPERIMENT NO. 19
Semester /Section:
Link to Code:
Date:
Faculty Signature:
Grade:
lxxxix
Machine
2022
Objective(s):
 Revise the concepts of supervised and unsupervised learning.

 Use feature selection to aid the solution to a real-world problem.
 Build a ML model from scratch and understand the issues involved.
 Perform extensive feature selection and dimensionality reduction.
Outcome:
Students will be able to apply the feature selection a and dimensionality reduction techniques
learned in ML and apply them to solve a real-world problem.
Problem Statement:
Build a ML model from scratch applying and comparing filter, wrapper, embedded and hybrid
feature selection approaches and dimensionality reduction.
Background Study:
In machine learning, algorithms are trained according to the data available and the research
question at hand. But if researchers fail to identify the useful features from the not so useful
features, tis process can take a lot of time and resources. To optimize model building and
selection, students should be able to understand and incorporate feature selection and
dimensionality reduction.
Question Bank:
1. What is the need of feature selection and dimensionality reduction?
2. What are the embedded methods of feature selection in ML?
Student Work Area

xc
Machine
2022
Annexure 2
Machine Learning
CSL313
Project Report
xci
Machine
2022
Faculty name: Student name:
Roll No.:
Semester:
Group:

The NorthCap University, Gurugram- 122001, India
Session 2022
Table of Contents
S.No Page
No.
1. Project Description
2. Problem Statement
xcii
Machine
2022
3. Analysis
3.1 Hardware Requirements
3.2 Software Requirements
4. Design
4.1 Data/Input Output Description:
4.2 Algorithmic Approach / Algorithm / DFD / ER

diagram/Program Steps
5. Implementation and Testing (stage/module

wise)
6. Output (Screenshots)
7. Conclusion and Future Scope

Aman19csu019 ML Lab Expierment 1,2,3,4,5,7

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Aman19csu019 ML Lab Expierment 1,2,3,4,5,7

Uploaded by

Copyright:

Available Formats

Machine Learning

Machine Learning Lab Manual

Department of Computer Science and Engineering

NorthCap University, Gurugram- 122001, India

School of Engineering and Technology

Department of Computer Science & Engineering

The NorthCap University Gurugram

• Laboratory Manual is for Internal Circulation only

No part of this Practical Record Book may be

reproduced, used, stored without prior permission of The NorthCap University

S.N. Details Page No.

5 List of Flip Assignment 15

8 Annexure 1 (Format of Lab Report) 18

2. Course Name: Machine Learning 3. Course Code 4. L-T-P 5. Credits

CO 3 Implement Recommendation System with real world data .

CO 4 Implement dimensionality reduction techniques to improve the efficiency of models

11. UNIT WISE DETAILS No. of Units: 5

Unit Number: 1 Title: Clustering No. of hours: 4

Unit Number: 3 Title: Recommendation System No. of hours:6

Unit Number: 4 Title: Dimensionality Reduction No. of hours:6

Unit Number: 5 Title: Model Selection and Boosting No. of hours:8

13. Course/Reference Books:

3. Tom Mitchell,”Machine Learning”, McGraw Hill, 1997.

Project (To be done as individual/in group): Individual

That ‘learning is a continuous process’ cannot be over emphasized. The theoretical

 Learning and understanding Association Rule Mining algorithms.

S.No. Requirements Details

3.1 General discipline in the lab

 Students should maintain silence while performing the experiments. If any

 Attendance in the lab class is compulsory.

3.3 Preparation and Performance

one dataset. Calculate the Silhouette Score. (Jupyter)

11. Python 4 CO4

5. LIST OF FLIP EXPERIMENTS

4 Marks: Observation & conduct of

Lab Practical Report

Faculty name: Dr. Vidhi Khanduja Student name: Aman Parkash

Roll No.: 19CSU019

Department of Computer Science and Engineering

4 Compare K-means and

5 Apply Apriori algorithm to

6 Apply Eclat algorithm to

7 Apply FP grow algorithm

Student Name and Roll Number: Aman Parkash and 19CSU019

Semester /Section: 6th / DS-E-9

 Understand what is unsupervised learning.

Students would be familiarized with unsupervised learning and clustering in particular.

1. What is the difference between supervised and unsupervised learning?

3. What are the different types of clustering approaches?

4. How to find the optimal value of k in k-means clustering?

Student Work Area

import seaborn as sns

import matplotlib.pyplot as plt

from sklearn.datasets import make_blobs

data = make_blobs(n_samples=200, n_features=2,

centers=4, cluster_std=1.8,random_state=10,return_centers = True)

# The number of features for each sample.

# The number of centers to generate, or the fixed center locations.

# If n_samples is an int and centers is None, 3 centers are generated.

# The standard deviation of the clusters

# shuffle bool, default=True

# used to shuffle the samples.

# Return the center od each cluster