UNIT-V

1
 UNIT-V
• Clustering: Introduction, Similarity and Distance Measures, Outliers,
Hierarchical Methods, Partitional Algorithms, Clustering Large
Databases, Clustering with Categorical Attributes, Comparison.
• Dimensionality Reduction: Linear Discriminant Analysis, Principal
Component Analysis
• Interactive Learning: Active Learning, Common heuristics, Sampling
bias, Safe Disagreement Based Active Learning Schemes
• Semi-Supervised Learning: Semi-supervised Learning, Transductive
SVM, Co-training
• Reinforcement Learning: Markov Decision Processes, Value
Iteration, Q-Learning.
2
 CLUSTERING
INTRODUCTION : Clustering is a technique
in machine learning that involves grouping
similar objects or data points into clusters based
on some similarity or distance measure.
There are various similarity and distance
measures that can be used in clustering, some of
the most commonly used ones are:
Euclidean distance: This is the most common
distance measure used in clustering. It calculates
the straight-line distance between two data
points in a n-dimensional space. This measure is
appropriate when the data points are continuous
and the features have the same unit of
measurement.
Euclidean distance: The Euclidean distance
between two n-dimensional data points x and y
is given by:
d(x, y) = √((x1 - y1)² + (x2 - y2)² + ... + (xn - yn)²)
where xi and yi are the values of the ith feature of
x and y, respectively.
Manhattan distance: Also known as the "city block distance", this
measure calculates the sum of the absolute differences between the
features of two data points. This measure is useful when the data is
in a grid-like structure, such as images or texts.
Manhattan distance: The Manhattan distance (L1 distance) between
two n-dimensional data points x and y is given by:
d(x, y) = |x1 - y1| + |x2 - y2| + ... + |xn - yn|
where xi and yi are the values of the ith feature of x and y,
respectively.
Cosine similarity: This measure calculates the cosine of the angle
between two data points in a multi-dimensional space. It is
commonly used in text and image processing, where the data is
represented as vectors.
Cosine similarity is a measure of similarity between two non-zero
n-dimensional vectors, and it is commonly used in text and image
processing applications. The formula for cosine similarity between
two vectors x and y is:
similarity(x, y) = (x . y) / (||x|| ||y||)
where x and y are two n-dimensional vectors, ||x|| and ||y|| are the
norms of x and y, respectively, and (x.y) is the dot product of x3 &
y.
The dot product of two vectors x and y is calculated as the
sum of the products of their corresponding components:
x . y = x1y1 + x2y2 + ... + xn*yn
The norm of a vector x is calculated as the square root of
the sum of the squares of its components:
||x|| = sqrt(x1^2 + x2^2 + ... + xn^2)
The resulting value of cosine similarity is a number
between -1 and 1, where values close to 1 indicate high
similarity, values close to -1 indicate high dissimilarity,
and a value of 0 indicates that the vectors are orthogonal or
have no similarity.
Pearson correlation coefficient: This measure is
commonly used in clustering of numerical data. It
calculates the correlation between the features of two data
points, taking into account the means and standard
deviations of the features.
Pearson correlation coefficient: The Pearson correlation
coefficient between two n-dimensional data points x and y
is given by:
corr(x, y) = (x - μx) . (y - μy) / (σx σy)
where μx and μy are the means of x and y, and σx and σy are
the standard deviations of x and y, respectively.
Jaccard similarity: This measure is used in
clustering of binary or categorical data. It calculates
the similarity between two data points based on the
number of common features they share.
Jaccard similarity: The Jaccard similarity between
two binary or categorical data points x and y is given
by:
sim(x, y) = |x ∩ y| / |x ∪ y|
where x ∩ y is the intersection of x and y, and x ∪ y
is the union of x and y.
These formulae can be used to compute the
similarity or distance between any two data points in
a dataset, which can then be used to group similar
data points together into clusters.
The choice of similarity or distance measure depends
on the type of data being clustered and the problem
at hand. It is important to choose a measure that is
appropriate for the data and the problem to obtain
meaningful results.
4
Hierarchical clustering is a method of clustering in which data
points are grouped together based on their similarity or
dissimilarity.
There are two main types of hierarchical clustering methods:
agglomerative and divisive.
Agglomerative hierarchical clustering starts with each data point
as a separate cluster and then iteratively merges the two closest
clusters into a new larger cluster until all data points are in a
single cluster.
This process can be visualized as a dendrogram, a tree-like
structure that shows the order in which the clusters were
merged.
Divisive hierarchical clustering, on the other hand, starts with
all data points in a single cluster and then iteratively splits the
cluster into two smaller clusters based on their dissimilarity,
until each data point is in a separate cluster.
Both agglomerative and divisive hierarchical clustering methods
require a measure of similarity or dissimilarity between data
points, as well as a method for determining the distance between
clusters.
The most common distance measures used in hierarchical
clustering are the Euclidean distance and the Manhattan
distance.
There are several methods for determining the
distance between clusters, including single
linkage, complete linkage, and average linkage.
Single linkage connects the closest points in two
clusters, while complete linkage connects the
farthest points in two clusters.
Average linkage computes the average distance
between all pairs of points in two clusters.
Hierarchical clustering is a useful method for
exploratory data analysis and can help to identify
natural groupings or patterns in the data.
However, it can be computationally expensive,
particularly for large datasets.
Additionally, the choice of distance measure and
clustering method can have a significant impact
on the results, and it can be difficult to interpret
the dendrogram when there are many clusters.
5
Partitioning algorithms are a class of algorithms used
in computer science and data analysis to divide a set of
data points into distinct groups or partitions based on
certain criteria.
These algorithms are commonly used in clustering and
classification tasks, and they aim to group data points
that are similar to each other while keeping different
groups as distinct as possible.
Here are some common types of partitioning
algorithms:
K-means algorithm: This is a popular clustering
algorithm that partitions data points into K clusters
based on their similarity to each other.
It is an iterative algorithm that minimizes the sum of
squared distances between data points and their
assigned cluster centroids.
Hierarchical clustering: This algorithm builds a tree-
like structure of clusters by recursively merging or
dividing data points based on their similarity.
The result is a hierarchical representation of the data,
which can be cut at different levels to obtain different
numbers of clusters.
Density-based clustering: This type of algorithm
identifies clusters based on regions of high data density,
rather than on explicit boundaries between clusters.
Points within a dense region are considered to be part of
the same cluster, while points in low-density regions are
considered outliers or noise.
Spectral clustering: This algorithm uses the
eigenvectors of a similarity matrix to partition data
points into clusters.
It is particularly useful when dealing with non-linearly
separable data, as it can capture the underlying manifold
structure of the data.
Partitioning Around Medoids (PAM): PAM algorithm
is similar to K-means, but instead of using cluster
centers (means), it uses actual data points (medoids) as
representatives of the clusters.
This makes PAM more robust to noise and outliers.
Overall, partitioning algorithms are powerful tools for
data analysis and can help reveal underlying patterns in
large datasets.
However, the choice of algorithm depends on the type of
data and the specific analysis goals.
6
CLUSTERING LARGE DATABASES is a common task
in data analysis and data mining.
The goal is to group similar data points together into
clusters, based on some similarity metric, so that it
becomes easier to analyze and understand the data.
Here are some common techniques and considerations for
clustering large databases:
Preprocessing: Before clustering, it is often necessary to
preprocess the data to remove noise, handle missing
values, and normalize the data.
Choosing a clustering algorithm: There are many
clustering algorithms available, including K-means,
hierarchical clustering, density-based clustering, and more.
The choice of algorithm depends on the specific
characteristics of the data and the goals of the analysis.
Choosing a distance metric: A distance metric measures
the similarity between two data points.
The choice of metric depends on the type of data being
analyzed and the characteristics of the clustering
algorithm.
Parallel processing: Clustering large databases can be
computationally intensive, so it may be necessary to use
parallel processing techniques to speed up the analysis.
Evaluation: It is important to evaluate the quality of
the clustering results to ensure that the clusters are
meaningful and useful.
This can be done by examining the clustering results
visually, or by using metrics such as silhouette score or
Rand index.
Iteration: Clustering can be an iterative process, where
the results of one clustering analysis are used to inform
the next iteration.
This can help to refine the clusters and improve the
overall quality of the analysis.
Scalability: As the size of the database grows, it may
become necessary to use distributed computing
techniques, such as MapReduce or Spark, to cluster the
data efficiently.
Overall, clustering large databases requires careful
consideration of the data, the clustering algorithm, and
the computational resources available.
With the right tools and techniques, it is possible to
gain valuable insights from even the largest and most
complex datasets.
7
 Partitioning-based clustering: Partitioning-based
Clustering with Categorical Attributes:
clustering algorithms, such as k-modes and k-
Clustering with categorical attributes can be challenging
prototypes, are specifically designed for clustering
because most clustering algorithms are designed to work with
categorical data.
numerical data.
These algorithms use a dissimilarity measure that is
However, there are several techniques that can be used to
based on the mode or prototype of each cluster.
cluster categorical data effectively:
Ensemble-based clustering: Ensemble-based
Convert categorical data to numerical data: One approach is
clustering algorithms, such as Co-Association Matrix
to convert categorical data to numerical data, using techniques
Clustering (COCA), can be used to cluster
such as one-hot encoding, binary encoding, or ordinal
categorical data by combining multiple clustering
encoding.
algorithms or similarity measures.
These techniques represent categorical data as a set of binary or
Visualization: Visualization techniques, such as
numerical features that can be used with traditional clustering
parallel coordinates plots and t-SNE, can be used to
algorithms.
visualize the categorical data in a low-dimensional
Similarity measures for categorical data: Another approach
space, making it easier to identify clusters and
is to use similarity measures that are specifically designed for
patterns.
categorical data.
Overall, clustering with categorical attributes
There are several similarity measures that can be used,
requires careful consideration of the data and the
including Jaccard similarity, Dice similarity, and Overlap
clustering algorithm or technique used. The choice of
similarity.
technique depends on the characteristics of the data
Hierarchical clustering: Hierarchical clustering can be used to
and the goals of the analysis. With the right tools and
cluster categorical data by computing a distance or similarity
techniques, it is possible to gain valuable insights
matrix between pairs of data points, using a similarity measure
from categorical data. 8
that is appropriate for categorical data.
COMPARISON: When clustering data with
categorical attributes, there are several techniques
that can be used, as mentioned in the previous
answer.
Here's a comparison of some of the most common
techniques:
Convert categorical data to numerical data: This
technique works well with clustering algorithms
that are designed to work with numerical data, such
as k-means and hierarchical clustering.
However, the technique can create high-
dimensional data, which can cause some algorithms
to perform poorly or require more computational
resources.
Similarity measures for categorical data:
Similarity measures that are designed for
categorical data can perform well, especially with
hierarchical clustering or partition-based clustering
algorithms.
However, the choice of similarity measure can have
a significant impact on the clustering results, and
some measures can be more appropriate than others
depending on the nature of the data.
Hierarchical clustering: Hierarchical clustering can be used with
both numerical and categorical data, but it can be more effective
with categorical data if a similarity measure that is appropriate for
categorical data is used.
Hierarchical clustering is also useful for identifying clusters at
different levels of granularity.
Partitioning-based clustering: Partition-based clustering
algorithms, such as k-modes and k-prototypes, are specifically
designed for clustering categorical data.
These algorithms can be very effective, especially when the data
has a large number of categorical variables.
However, the algorithms can be more computationally intensive
and require more resources than other clustering techniques.
Ensemble-based clustering: Ensemble-based clustering
algorithms can be effective when different clustering techniques
or similarity measures are combined.
These algorithms can be more robust to the choice of clustering
technique or similarity measure, but they can also be more
complex to implement.
Visualization: Visualization techniques can be used to explore the
data and identify clusters. However, visualization techniques can
be subjective and can be affected by the choice of visualization
technique or the dimensionality reduction technique used. 9
Overall, the choice of clustering technique depends on the
nature of the data and the goals of the analysis.
It may be useful to experiment with different techniques and
compare the results to determine which approach is most
effective for the particular dataset and clustering problem.
Dimensionality reduction is the process of reducing the
number of variables or features in a dataset while retaining
as much of the important information as possible.
This is often done to simplify the data and make it easier to
analyze, visualize, and interpret.
Dimensionality reduction can be particularly useful when
working with high-dimensional data, where the number of
features can be very large.
There are two main approaches to dimensionality reduction:
feature selection and feature extraction.
Feature selection: Feature selection involves selecting a
subset of the original features and discarding the rest.
This approach can be based on various criteria such as
variance, correlation, or mutual information.
The main advantage of feature selection is that it can be
simple and efficient, but it may result in loss of important
information that is not captured by the selected features.
Feature extraction: Feature extraction involves
creating new features that are a combination of the
original features.
This approach can be done through techniques such
as Principal Component Analysis (PCA), Linear
Discriminant Analysis (LDA), or t-distributed
Stochastic Neighbor Embedding (t-SNE).
Feature extraction can be more effective than feature
selection in preserving important information while
still reducing the dimensionality of the data.
However, it can also be more complex and
computationally intensive.
Some common techniques for dimensionality
reduction include:
Principal Component Analysis (PCA): PCA is a
widely used technique that involves finding the linear
combinations of the original features that explain the
most variance in the data.
The resulting components are then used as the new
features.
10
Linear Discriminant Analysis (LDA): LDA is a supervised
technique that seeks to find a linear combination of the
original features that maximizes the separation between
classes.
This approach is useful for classification problems.
t-distributed Stochastic Neighbor Embedding (t-SNE):
t-SNE is a non-linear dimensionality reduction technique
that is particularly useful for visualizing high-dimensional
data in a low-dimensional space.
Overall, dimensionality reduction is a powerful tool for
simplifying and analyzing complex data.
The choice of technique depends on the nature of the data,
the goals of the analysis, and the computational resources
available.
Linear Discriminant Analysis (LDA) is a supervised
learning algorithm that is used for dimensionality reduction
and classification. LDA is used when we have labeled data
and want to find the best linear combination of the features
that separates the classes in the data.
The main idea behind LDA is to find a linear transformation
of the features that maximizes the separation between
classes.
This transformation is done in such a way that the new
features have the highest possible between-class
variance and the lowest possible within-class variance.
The basic steps involved in LDA are as follows:
Standardize the data: LDA assumes that the features
are normally distributed, so we need to standardize the
data to have zero mean and unit variance.
Compute the class means and overall mean:
Compute the mean of each feature for each class and
the mean of each feature overall.
Compute the between-class and within-class scatter
matrices: Compute the scatter matrix for each class
and the scatter matrix for the overall data.
Compute the eigenvalues and eigenvectors of the
generalized eigenvalue problem: The eigenvectors
are used as the new features.
Select the top k eigenvectors: Select the top k
eigenvectors that correspond to the k largest
eigenvalues. These eigenvectors are the new features
that are used for classification.
LDA can be used for both binary and multi-class
classification problems. In the binary case, LDA finds
11
a line that separates the two classes.
In the multi-class case, LDA finds a hyperplane that
separates the classes.
Compute the eigenvalues and eigenvectors of the
generalized eigenvalue problem: The eigenvectors are
used as the new features.
Select the top k eigenvectors: Select the top k eigenvectors
that correspond to the k largest eigenvalues.
These eigenvectors are the new features that are used for
classification.
LDA can be used for both binary and multi-class
classification problems.
In the binary case, LDA finds a line that separates the two
classes.
In the multi-class case, LDA finds a hyperplane that
separates the classes.
LDA has several advantages, including its simplicity and
effectiveness for classification problems with well-separated
classes.
However, LDA assumes that the classes have equal
covariance matrices, which may not be true in practice.
LDA also assumes that the data is normally distributed,
which may not be the case for all datasets.
Linear Discriminant Analysis (LDA) is a
supervised machine learning algorithm that can be
used for classification problems.
The goal of LDA is to find a linear combination of
features that maximally separates different classes in
the data. Here is the mathematical formula for LDA:
Assuming that we have two classes in our data,
labeled as class 0 and class 1, and the input features
are represented by a d-dimensional vector x, LDA
tries to find a projection vector w such that the
projected data w'x is well-separated by class.
The mean of each class, denoted as μ0 and μ1,
respectively, can be computed as follows:
μ0 = (1/n0) * Σxi for all i such that yi = 0
μ1 = (1/n1) * Σxi for all i such that yi = 1
where xi is the ith data point and yi is the label of
that data point (either 0 or 1). n0 and n1 are the
number of data points in class 0 and class 1,
respectively.
The scatter matrix within each class, denoted as Sw,
can be computed as follows:
12 y =
Sw = Σi(xi - μi)(xi - μi)' for all i such that yi = 0 or i
The scatter matrix between classes, denoted as
Sb, can be computed as follows: Sb = (μ1 - μ0)(μ1
- μ0)'
The LDA projection vector w can be obtained by
solving the following optimization problem:
maximize J(w) = (w'Sb w) / (w'Sw w)
subject to ||w|| = 1, where ||w|| represents the
Euclidean norm of w.
The optimal projection vector w can be found
using eigenvalue decomposition of the matrix
Sw^-1Sb.
The direction of the eigenvector corresponding
to the largest eigenvalue is the optimal projection
direction.
Finally, the projected data can be obtained by
computing the dot product of the data points and
the projection vector:
y = w'x.
The classification rule can then be based on the
sign of y: if y is positive, the data point belongs
to class 1, otherwise it belongs to class 0.
Principal Component Analysis (PCA) is a statistical
technique used to reduce the dimensionality of a dataset.
It is a method of transforming the data to a new
coordinate system in such a way that the first coordinate
(or principal component) captures the maximum amount
of variation in the data, the second coordinate captures
the maximum amount of remaining variation, and so on.
PCA works by identifying the linear combinations of the
original variables that account for the most variance in
the data.
These linear combinations are known as principal
components.
The first principal component is the linear combination
that accounts for the most variance in the data, the
second principal component is the linear combination
that accounts for the most remaining variance, and so
on.
PCA can be used for a variety of purposes, including
data visualization, feature extraction, and noise
reduction.
13
It is often used in machine learning to preprocess data
before applying other algorithms.
By reducing the dimensionality of the data, PCA can also
help to reduce the computational complexity of
subsequent algorithms.
PCA assumes that the data is normally distributed and
linearly related.
It can also be sensitive to outliers, and it is important to
scale the data before applying PCA to ensure that each
variable is given equal weight in the analysis.
Principal Component Analysis (PCA) is a statistical
technique that involves finding the principal components
of a dataset.
The principal components are the directions of maximum
variation in the data. Here is the mathematical formula for
PCA:
Assuming that we have n observations, each with p
features, we can represent the dataset as an n x p matrix
X. We first center the data by subtracting the mean of
each feature from each observation:
Z = X - 1nX
where 1n is an n x n matrix of ones.
The covariance matrix of the centered data is given by:
C = 1/(n-1) Z'Z
We then calculate the eigenvectors and eigenvalues of C.
The eigenvectors represent the principal components of
the data, while the eigenvalues represent the amount of
variance explained by each principal component.
The first principal component, denoted by PC1, is the
eigenvector corresponding to the largest eigenvalue.
The second principal component, denoted by PC2, is the
eigenvector corresponding to the second largest
eigenvalue, and so on.
The principal components can be used to transform the
original data into a new coordinate system.
The transformed data, denoted by Y, is given by:
Y = ZX
where Z is the matrix of principal components, with each
column representing a principal component.
The amount of variance explained by each principal
component can be calculated as the ratio of its eigenvalue
to the sum of all eigenvalues.
This can be used to determine how many principal
components to retain for a given application. 14
PCA can also be used for dimensionality reduction, by
retaining only the top k principal components, where k is a
smaller number than the original number of features p.
The transformed data Y can then be used as a lower-
dimensional representation of the original data X.
INTERACTIVE LEARNING: Active Learning
Active learning is a type of interactive learning where the
algorithm is allowed to choose which data samples to learn
from in order to improve its performance.
Unlike traditional supervised learning, where the algorithm
is trained on a fixed set of labeled data, in active learning,
the algorithm is allowed to select the most informative
samples from an unlabeled dataset and request labels for
those samples.
The labeled samples are then used to update the model and
the process is repeated iteratively.
The key idea behind active learning is to select the most
informative samples to label, rather than labeling random
samples. This is done by selecting samples that the model is
uncertain about or that are representative of the underlying
distribution.
By doing so, active learning can achieve high
performance with fewer labeled samples compared to
traditional supervised learning.
There are different active learning strategies that can
be used to select the most informative samples.
Some common strategies include uncertainty
sampling, query by committee, and density-based
sampling.
Uncertainty sampling: This strategy selects samples
that the model is uncertain about.
For example, in a binary classification problem, the
algorithm may select samples that are close to the
decision boundary, as these samples are likely to be
the most difficult to classify.
Query by committee: This strategy involves training
multiple models on the same dataset and selecting
samples that the models disagree on.
This is based on the assumption that the samples that
are difficult for one model to classify are also difficult
for other models.
15
Density-based sampling: This strategy selects samples
from regions of the feature space that are underrepresented
in the labeled data.
This can be done using clustering techniques or other
density-based methods.
Overall, active learning can be a powerful tool for machine
learning when labeled data is limited or expensive to obtain.
By selecting the most informative samples to label, active
learning can improve model performance with fewer labeled
samples, making it an efficient and effective way to learn
from data.
Interactive Learning: common heuristics
Interactive learning involves a range of techniques and
heuristics that can be used to optimize the learning process.
Here are some common heuristics used in interactive
learning:
Active exploration: This heuristic involves exploring the
environment to gather data and build a model.
It is commonly used in reinforcement learning, where an
agent interacts with the environment to learn the best
actions to take in a given situation.
Uncertainty sampling: As mentioned earlier,
uncertainty sampling is a common heuristic used in
active learning.
It involves selecting the samples that the model is most
uncertain about, in order to improve performance.
Query-by-committee: This heuristic involves using
multiple models to evaluate the uncertainty of different
samples.
Samples that are uncertain across multiple models are
considered more informative and are given priority for
labeling.
Diversity sampling: This heuristic involves selecting
samples that are representative of different parts of the
data distribution.
This can help to avoid biasF in the training data and
improve model performance on new, unseen data.
Random sampling: This heuristic involves selecting
samples at random from the dataset.
While not as effective as other heuristics, random
sampling can be useful for building a baseline model or
for exploring the dataset.
16
Active learning with expert feedback: In this heuristic, an
expert provides feedback to guide the selection of
informative samples.
This can be useful in situations where the model is being
used to solve a specific task and where the expert has domain
knowledge that can be leveraged to improve performance.
These are just a few of the common heuristics used in
interactive learning.
The choice of heuristic depends on the specific problem and
the data available, and often involves a trade-off between
exploration and exploitation, diversity and uncertainty, and
other factors.
Interactive Learning: Sampling bias
Sampling bias is a common issue in machine learning, and it
can also affect interactive learning.
Sampling bias occurs when the examples that are used to
train or test the model are not representative of the
population that the model will be applied to.
In interactive learning, sampling bias can occur when the
examples that are presented to the user for labeling or
feedback are not representative of the population that the
model will be applied to.
Sampling bias can lead to inaccurate or incomplete
models, as the model is trained on a limited and
biased set of data.
For example, if the examples presented to the user for
labeling are biased towards a particular demographic
group, the model may not perform well on data from
other demographic groups.
Similarly, if the examples presented to the user for
labeling are biased towards a particular set of features
or characteristics, the model may not perform well on
data that contains different features or characteristics.
To avoid sampling bias in interactive learning, it is
important to ensure that the examples presented to the
user for labeling or feedback are representative of the
population that the model will be applied to. This can
be achieved by carefully selecting the initial set of
examples, or by using sampling strategies that ensure
diversity and balance in the examples presented to the
user. It is also important to monitor the performance
of the model over time, and to re-evaluate the
sampling strategy as new data becomes available.
17
By addressing sampling bias in interactive learning, we
can help to ensure that the resulting model is accurate,
reliable, and robust in its performance.
Safe Disagreement Based Active Learning
Schemes:
Safe Disagreement Based Active Learning (SDAL) is a
type of interactive learning that aims to select
informative examples for the user to label or provide
feedback on, while also ensuring that the model's
predictions are safe and reliable.
SDAL is particularly useful in scenarios where incorrect
predictions could have serious consequences, such as in
healthcare or finance.
The basic idea behind SDAL is to select examples on
which the model's predictions are uncertain, but on
which there is high agreement between different versions
of the model.
The idea is that by selecting these examples, the user can
provide feedback that will help to improve the model's
accuracy and reduce its uncertainty, while also ensuring
that the model's predictions are safe and reliable.
There are several different schemes that can be used to
implement SDAL. One common approach is to use an
ensemble of models, each of which is trained on a
different subset of the data.
By selecting examples on which there is high
disagreement between the different models, we can
identify examples that are particularly challenging for the
model and that are likely to benefit from user feedback.
Another approach is to use a Bayesian model, which
allows for uncertainty in the model's predictions.
By selecting examples on which the model's uncertainty
is high, we can identify examples that are particularly
informative for improving the model's accuracy.
Overall, SDAL is a powerful approach to interactive
learning that can help to improve the accuracy and
reliability of machine learning models in high-stakes
applications.
By selecting examples that are both informative and safe,
we can ensure that the resulting model is well-suited to its
intended use and can be relied upon to make accurate and
trustworthy predictions.
18
Semi-supervised learning is a type of machine learning
that falls between the two main paradigms of supervised
learning and unsupervised learning.
In supervised learning, the model is trained using labeled
examples, where the correct outputs are known.
In unsupervised learning, the model is trained using
unlabeled examples, where the correct outputs are not
known.
Semi-supervised learning, on the other hand, combines
both labeled and unlabeled data to train a model.
The basic idea behind semi-supervised learning is that
there is often much more unlabeled data available than
labeled data.
By using both types of data, we can take advantage of the
large amounts of unlabeled data to improve the accuracy
of the model, while still benefiting from the labeled data
to guide the learning process.
Semi-supervised learning algorithms typically work by
first training an unsupervised model on the unlabeled
data.
This unsupervised model is then used to generate labels
for the unlabeled data, which are then combined with the
labeled data to train a supervised model.
The supervised model is then fine-tuned using both the
labeled and unlabeled data, with the goal of improving
the model's accuracy on the labeled data.
One of the main advantages of semi-supervised learning
is that it can help to reduce the cost and effort of labeling
large amounts of data, which can be time-consuming and
expensive.
It can also help to improve the accuracy of the resulting
model, as the model is trained on a larger and more
diverse set of data.
However, semi-supervised learning can also be
challenging, as it requires careful selection of the
unlabeled data and the unsupervised model used to
generate labels.
In addition, semi-supervised learning algorithms can be
more complex and computationally intensive than
supervised learning algorithms, as they require additional
steps to process and integrate the unlabeled data.
Overall, semi-supervised learning is a powerful approach
to machine learning that can help to improve the accuracy
and efficiency of models, particularly in scenarios where
labeled data is scarce or expensive to obtain. 19
Semi-Supervised Learning: Transductive
SVM:
Transductive Support Vector Machines (TSVMs) are
a type of semi-supervised learning algorithm that
combines ideas from supervised and unsupervised
learning to make predictions on a set of unlabeled
data.
TSVMs are particularly useful when we have a small
amount of labeled data and a much larger amount of
unlabeled data.
The basic idea behind TSVMs is to use the labeled
data to train a Support Vector Machine (SVM) model,
which is a supervised learning algorithm used for
classification or regression.
The SVM model learns a decision boundary that
separates the different classes in the labeled data.
This decision boundary is then used to predict the
labels of the unlabeled data, which is done in a
transductive manner, meaning that the predictions are
only made on the specific set of unlabeled examples
that are given.
TSVMs work by considering all possible labelings of the
unlabeled data, and then selecting the labeling that results in
the largest margin between the decision boundary and the data
points of each class.
This is done using an iterative optimization procedure, which
alternates between optimizing the SVM model on the labeled
data, and updating the labeling of the unlabeled data based on
the current decision boundary.
One of the main advantages of TSVMs is that they can be
very effective in scenarios where there is a large amount of
unlabeled data and only a small amount of labeled data.
By using the labeled data to guide the learning process and
the unlabeled data to improve the accuracy of the model,
TSVMs can achieve better performance than fully supervised
learning algorithms that rely only on the labeled data.
However, TSVMs can also be computationally expensive, as
they require the optimization of both the SVM model and the
labeling of the unlabeled data.
In addition, TSVMs assume that the labeled data is
representative of the distribution of the unlabeled data, which
may not always be the case in practice.
20
Overall, TSVMs are a powerful and flexible
approach to semi-supervised learning, which can
be used in a variety of applications where labeled
data is scarce or expensive to obtain.
CO-TRAINING:
Co-training is another popular approach to semi-
supervised learning that can be used to improve
the accuracy of a machine learning model by
leveraging both labeled and unlabeled data.
Like TSVMs, co-training is particularly useful when
we have a small amount of labeled data and a
much larger amount of unlabeled data.
The basic idea behind co-training is to train
multiple models, each using a different set of
features, or views, of the data.
These views may be obtained, for example, by
applying different feature extraction techniques or
by using different subsets of the available
features.
Each model is trained on a different subset of the
labeled data, and the unlabeled data is then used
to iteratively improve the accuracy of both
models.
During each iteration, the two models make predictions on the
unlabeled data, and the predictions that are most confident and
consistent between the two models are used to augment the
labeled data for the next iteration.
This process of selecting the most informative examples and
adding them to the labeled data is known as "pool-based active
learning."
The two models are then retrained on the expanded labeled data,
and the process is repeated until the accuracy of both models
converges.
Co-training can be a very effective approach to semi-supervised
learning, as it allows us to leverage the complementary
information in multiple views of the data to improve the accuracy
of the model.
By selecting the most informative examples from the unlabeled
data and adding them to the labeled data, co-training can also help
to reduce the amount of labeled data required to achieve high
accuracy.
One of the main limitations of co-training is that it requires the
data to be easily separable into two or more views, which may not
always be the case in practice. In addition, co-training may not be
as effective in scenarios where the unlabeled data is very noisy or
21
contains irrelevant information.
Overall, co-training is a powerful and flexible approach to
semi-supervised learning that can be used in a variety of
applications where labeled data is scarce or expensive to
obtain.
When combined with other semi-supervised learning
techniques, such as TSVMs, co-training can help to further
improve the accuracy and efficiency of machine learning
models.
Reinforcement Learning (RL)
Reinforcement Learning (RL) is a type of machine learning
that deals with sequential decision-making problems.
In RL, an agent interacts with an environment, takes actions
based on the current state, and receives feedback in the form of
rewards.
The goal of the agent is to learn a policy, which is a mapping
from states to actions that maximizes the expected cumulative
reward over time.
The RL framework is similar to how humans learn through
trial and error. For example, when a child learns to walk, they
take small steps and adjust their movements based on the
feedback received from their environment.
Similarly, in RL, an agent takes actions and adjusts its policy
based on the rewards received from the environment.
RL is used in a wide range of applications, such as
robotics, gaming, and finance, to model decision-
making problems in which the consequences of
actions are uncertain and the goal is to maximize long-
term rewards.
RL has been successfully applied in many real-world
scenarios, such as training autonomous vehicles,
optimizing energy consumption in buildings, and
developing personalized medical treatments.
The RL framework consists of four main components:
the environment, the agent, the state, and the reward.
The environment represents the problem being solved,
and it provides feedback to the agent in the form of
rewards.
The agent takes actions based on the current state and
the policy, and it receives feedback from the
environment in the form of rewards.
The state represents the current situation of the
environment, and it is used by the agent to make
decisions.
The reward is a scalar signal that indicates the quality
of the action taken by the agent.
22
RL algorithms can be broadly classified into model-
based and model-free methods. Model-based methods
use a model of the environment to predict the outcomes
of actions, while model-free methods directly estimate
the value of actions without modeling the environment.
Some popular RL algorithms include Q-learning, Deep
Q-Networks (DQNs), and Policy Gradients.
Overall, RL is a powerful and flexible approach to
solving sequential decision-making problems.
By learning from experience and optimizing long-term
rewards, RL can help to find optimal solutions in
complex and uncertain environments.
Markov Decision Processes (MDPs) are a formal
framework used in Reinforcement Learning to model
sequential decision-making problems. An MDP is
defined as a tuple (S, A, P, R, γ), where:
S is a set of states.
A is a set of actions.
P is the transition function, which specifies the
probability of transitioning from one state to another
when taking an action.
R is the reward function, which specifies the immediate
reward received after taking an action in a state.
γ is the discount factor, which determines the importance
of future rewards.
MDPs are characterized by the Markov property, which
states that the future state and reward depend only on the
current state and action, and not on the history of
previous states and actions.
This property allows us to represent a complex decision-
making problem as a simple mathematical framework.
In an MDP, an agent takes actions based on the current
state, receives a reward, and transitions to a new state
according to the transition function.
The goal of the agent is to learn a policy, which is a
mapping from states to actions that maximizes the
expected cumulative reward over time.
This is known as the "optimal policy.“
The optimal policy can be found using various methods,
such as Dynamic Programming, Monte Carlo methods,
and Temporal Difference learning.
23
These methods allow the agent to learn the values of
different states and actions, which can be used to guide the
selection of actions that maximize the expected cumulative
reward.
MDPs are used in a wide range of applications, such as
robotics, gaming, and finance, to model decision-making
problems in which the consequences of actions are uncertain
and the goal is to maximize long-term rewards.
By using the mathematical framework of MDPs, we can
formalize these problems and develop efficient algorithms to
find optimal solutions.
VALUE ITERATION:
Value Iteration is a Dynamic Programming algorithm used
in Reinforcement Learning to find the optimal value
function and the optimal policy for a Markov Decision
Process (MDP).
It is an iterative algorithm that updates the value of each
state using the Bellman equation.
The value of a state in an MDP is defined as the expected
cumulative reward starting from that state and following the
optimal policy.
The Bellman equation expresses the value of a state
in terms of the values of its neighboring states:
V(s) = max_a ∑_s' P(s'|s,a) [R(s,a,s') + γ V(s')]
where V(s) is the value of state s, a is the action
taken in state s, s' is the next state, P(s'|s,a) is the
probability of transitioning to s' when taking action a
in state s, R(s,a,s') is the reward received when
transitioning from state s to s' with action a, and γ is
the discount factor.
The Value Iteration algorithm starts with an initial
estimate of the value function and iteratively updates
the values of all states until convergence.
At each iteration, the algorithm updates the value of
each state using the Bellman equation:
V_k+1(s) = max_a ∑_s' P(s'|s,a) [R(s,a,s') + γ
V_k(s')]
where V_k(s) is the value of state s at the k-th
iteration. 24
The algorithm continues to iterate until the values of all
states converge to their optimal values.
Once the optimal value function is obtained, the optimal
policy can be derived by choosing the action that maximizes
the value of each state:
π*(s) = argmax_a ∑_s' P(s'|s,a) [R(s,a,s') + γ V*(s')]
where π*(s) is the optimal policy in state s and V*(s) is the
optimal value of state s.
Value Iteration is a powerful algorithm for solving MDPs,
but it can be computationally expensive for large state
spaces. In such cases, other methods like Policy Iteration or
Monte Carlo methods may be more suitable.
Q-LEARNING
Q-Learning is a model-free Reinforcement Learning
algorithm used to learn the optimal action-value function for
a Markov Decision Process (MDP).
The action-value function, denoted as Q(s,a), represents the
expected cumulative reward for taking action a in state s and
following the optimal policy thereafter.
Q-Learning uses an iterative approach to update the Q-
values for each state-action pair based on the observed
rewards.
The Q-Learning algorithm uses an exploration-
exploitation trade-off strategy to balance between
exploiting the known information about the
environment and exploring new actions to gain more
information.
The algorithm maintains a table of Q-values for each
state-action pair, and it starts with an initial estimate of
the Q-values.
At each time step, the agent selects an action based on
the current state and the Q-values, using an
exploration strategy like Epsilon-greedy or Softmax.
After taking the action, the agent observes the next
state and the reward and updates the Q-value for the
current state-action pair using the following equation:
Q(s,a) ← Q(s,a) + α [r + γ max_a' Q(s',a') - Q(s,a)]
25
where s is the current state, a is the action taken, r is the
reward received, s' is the next state, a' is the next action, γ is
the discount factor that determines the importance of future
rewards, and α is the learning rate that controls the weight of
the new information.
The Q-Learning algorithm iteratively updates the Q-values
based on the observed rewards, and the Q-values converge
to the optimal values for each state-action pair.
Once the optimal Q-values are obtained, the optimal policy
can be derived by choosing the action with the highest Q-
value in each state.
Q-Learning is a simple and powerful algorithm that can be
applied to a wide range of Reinforcement Learning
problems.
It has been successfully used in many real-world
applications, such as training autonomous agents, optimizing
control systems, and playing games like chess and Go.
However, Q-Learning may suffer from slow convergence
and suboptimal results when the state space is large or the
reward function is sparse.
In such cases, other algorithms like Deep Q-Networks
(DQNs) or Monte Carlo methods may be more suitable.
26