CCA42006 – MACHINE LEARNING

MCA – Semester 2
Anitha S Pillai
Professor
School of Computing Sciences,
Department of Computer Applications
 Unit1

• Learning
•Types of machine learning - Supervised learning
• The brain and the neurons
• Linear Discriminants
•Perceptron
• Linear Separability
•Linear Regression
•Multilayer perceptron
•Examples of using MLP
• Back propagation of error
 What is Machine Learning

• Machine learning is an application of artificial intelligence (AI) that provides systems the ability to
automatically learn and improve from experience without being explicitly programmed.
• •Machine learning focuses on the development of computer programs that can access data and use it
learn for themselves.
 Mitchell’s Machine Learning(ML)

• ML is concerned with the question of how to construct computer programs that

automatically improve with experience.
• •A computer program is said to learn from experience E with respect to some class of tasks
T and performance measure P, if its performance at tasks in T, as measured by P, improves
with experience E.
 Example

• In our spam/non-spam example, the examples (E) are emails .

• •The task (T) was a decision problem (called classification) of marking each email as spam or not, and
putting it in the correct folder.
• •Our performance measure (P) would be something like accuracy as a percentage (correct decisions
divided by total decisions made multiplied by 100) between 0% (worst) and 100% (best).
Types of Learning
 Supervised Learning

• Supervised learning is when the model is getting trained on a labelled dataset

• Labelled dataset is one which have both input and output parameters
• In this type of learning both training and validation datasets are labelled
Supervised Learning
 Classification

• Classification : It is a Supervised Learning task where output is having defined labels(discrete value)
• • For example – Purchased has defined labels i.e. 0 or 1 ; 1 means the customer will purchase and 0
means that customer won’t purchase
• •The goal here is to predict discrete values belonging to a particular class and evaluate on the basis of
accuracy.
• •It can be either binary or multi class classification.
• • In binary classification, model predicts either 0 or 1 ; yes or no but in case of multi class
classification, model predicts more than one class.
• •Example: Gmail classifies mails in more than one classes like social, promotions, updates, forum.
 Regression

• Regression : It is a Supervised Learning task where output is having continuous value.

• •Example Wind Speed is not having any discrete value but is continuous in the particular range.
• •The goal here is to predict a value as much closer to actual output value as our model can and then
evaluation is done by calculating error value.
• •The smaller the error the greater the accuracy of our regression model.
 Examples of Supervised Learning Algorithm

• Linear Regression
• •Nearest Neighbor
• •Guassian Naive Bayes
• •Decision Trees
• •Support Vector Machine (SVM)
• •Random Forest
 UnSupervised Learning Algorithm

• Unsupervised learning is where you only have input data (X) and no corresponding output variables.
• •These are called unsupervised learning because unlike supervised learning above there is no correct
answers and there is no teacher.
• •Algorithms are left to their own devises to discover and present the interesting structure in the data
 UnSupervised Learning Algorithm

• Unsupervised learning problems can be further grouped into clustering and association problems.
• •Clustering: A clustering problem is where you want to discover the inherent groupings in the data,
such as grouping customers by purchasing behavior.
• •Association: An association rule learning problem is where you want to discover rules that describe
large portions of your data, such as people that buy X also tend to buy Y.
• •Some popular examples of unsupervised learning algorithms are:
• •k-means for clustering problems.
• •Apriori algorithm for association rule learning problems.
 Semi Supervised Learning Algorithm

• Problems where you have a large amount of input data (X) and only some of the data is labeled (Y) are
called semi-supervised learning problems.
• •These problems sit in between both supervised and unsupervised learning.
• •A good example is a photo archive where only some of the images are labeled, (e.g. dog, cat, person) and
the majority are unlabeled.
• •It can be expensive or time-consuming to label data as it may require access to domain experts. Whereas
unlabeled data is cheap and easy to collect and store.
• •You can use unsupervised learning techniques to discover and learn the structure in the input variables.
• •You can also use supervised learning techniques to make best guess predictions for the unlabeled data,
feed that data back into the supervised learning algorithm as training data and use the model to make
predictions on new unseen data.
 Reinforcement Learning Algorithm

• Reinforcement learning is an area of Machine Learning

• •It is about taking suitable action to maximize reward in a particular situation
• • It is employed by various software and machines to find the best possible behavior or path it should
take in a specific situation
• • Reinforcement learning differs from the supervised learning in a way that in supervised learning the
training data has the answer key with it so the model is trained with the correct answer itself whereas
in reinforcement learning, there is no answer but the reinforcement agent decides what to do to
perform the given task.
• •In the absence of training dataset, it is bound to learn from its experience.
 Linear Discriminant Analysis

• A dimensionality reduction technique which is commonly used for the supervised classification
problems.
• •It is used for modeling differences in groups i.e. separating two or more classes.
• •For example, we have two classes and we need to separate them efficiently.
• •Classes can have multiple features.
• •Using only a single feature to classify them may result in some overlapping.
• •So, we will keep on increasing the number of features for proper classification.
 Linear Discriminant Analysis

• Suppose we have two sets of data points belonging to two different classes that we want to classify.
• • when the data points are plotted on the 2D plane, there’s no straight line that can separate the two
classes of the data points completely.
• •Hence, in this case, LDA (Linear Discriminant Analysis) is used
Linear Discriminant Analysis
Biological Neuron
 Biological Neuron

• A human brain has billions of neurons.

• •Neurons are interconnected nerve cells in the human brain that are involved in processing and
transmitting chemical and electrical signals.
• •Dendrites are branches that receive information from other neurons.
• •Cell nucleus or Soma processes the information received from dendrites.
• •Axon is a cable that is used by neurons to send information.
• •Synapse is the connection between an axon and other neuron dendrites.
 Applications

• Face Recognition: In the field of Computer Vision, face recognition is a very popular application in
which each face is represented by a very large number of pixel values.
• •Linear discriminant analysis (LDA) is used here to reduce the number of features to a more
manageable number before the process of classification.
 Applications

• Medical: In this field, Linear discriminant analysis (LDA) is used to classify the patient disease state as
mild, moderate or severe based upon the patient various parameters and the medical treatment he is
going through.
• •This helps the doctors to intensify or reduce the pace of their treatment.
 Applications

• Customer Identification: Suppose we want to identify the type of customers which are most likely to
buy a particular product in a shopping mall.
• •By doing a simple question and answers survey, we can gather all the features of the customers.
• •Here, Linear discriminant analysis will help us to identify and select the features which can describe
the characteristics of the group of customers that are most likely to buy that particular product in the
shopping mall.
 Perceptron
• Takes an input, aggregates it (weighted sum) and returns 1 only if the aggregated sum is
 Linear Separability

https://en.wikipedia.org/wiki/Linear_separability
 Linear Separability
• Linear separability implies that if there are two classes then there will be a point,
line, plane, or hyperplane that splits the input features in such a way that all points
of one class are in one-half space and the second class is in the other half-space

https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781788830577/2/ch02lvl1sec26/linear-separability
 Linear Regression

• Linear regression is a linear model, e.g. a model that assumes a linear relationship between the input
variables (x) and the single output variable (y)
• More specifically, that y can be calculated from a linear combination of the input variables (x)

• When there is a single input variable (x), the method is referred to as simple linear regression
• When there are multiple input variables, literature from statistics often refers to the method as
multiple linear regression
 Linear Regression

• Linear regression is a linear model, e.g. a model that assumes a linear relationship between the input
variables (x) and the single output variable (y)
• More specifically, that y can be calculated from a linear combination of the input variables (x)

• When there is a single input variable (x), the method is referred to as simple linear regression
• When there are multiple input variables, literature from statistics often refers to the method as
multiple linear regression

y = B0 + B1*x
 Multilayer Perceptron

• The field of artificial neural networks is often just called neural networks or multi-layer perceptrons
after perhaps the most useful type of neural network
• A perceptron is a single neuron model that was a precursor to larger neural networks.
• It is a field that investigates how simple models of biological brains can be used to solve difficult
computational tasks like the predictive modeling tasks we see in machine learning
• The goal is not to create realistic models of the brain, but instead to develop robust algorithms and
data structures that we can use to model difficult problems
 Multilayer Perceptron

• The power of neural networks comes from their ability to learn the representation in your training
data and how to best relate it to the output variable that you want to predict
• In this sense neural networks learn a mapping
• Mathematically, they are capable of learning any mapping function and have been proven to be a
universal approximation algorithm.
• The predictive capability of neural networks comes from the hierarchical or multi-layered structure of
the networks
• The data structure can pick out (learn to represent) features at different scales or resolutions and
combine them into higher-order features
• For example from lines, to collections of lines to shapes.
 Multilayer Perceptron

• A multi-layered perceptron (MLP) is one of the most common neural network models used in the
field of deep learning
• Often referred to as a “vanilla” neural network, an MLP is simpler than the complex models of
today’s era
• However, the techniques it introduced have paved the way for further advanced neural networks.
• The multilayer perceptron (MLP) is used for a variety of tasks, such as stock analysis, image
identification, spam detection, and election voting predictions.
 Multilayer Perceptron

• A multi-layered perceptron consists of interconnected neurons transferring information to each other,

much like the human brain. Each neuron is assigned a value. The network can be divided into three
main layers
• Input Layer
This is the initial layer of the network which takes in an input which will be used to produce an output
• Hidden Layer(s)
• The network needs to have at least one hidden layer. The hidden layer(s) perform computations and
operations on the input data to produce something meaningful.
 Multilayer Perceptron

• Output Layer
• The neurons in this layer display a meaningful output.

• Connections
• The MLP is a feedforward neural network, which means that the data is transmitted from the input
layer to the output layer in the forward direction.

• The connections between the layers are assigned weights

• The weight of a connection specifies its importance
• This concept is the backbone of an MLP’s learning process.
Multilayer Perceptron
 Backpropagation
• Backpropagation is a technique used to optimize the weights of an MLP using the outputs as inputs

• In a conventional MLP, random weights are assigned to all the connections

• These random weights propagate values through the network to produce the actual output
• Naturally, this output would differ from the expected output
• The difference between the two values is called the error
• Backpropagation refers to the process of sending this error back through the network, readjusting
the weights automatically so that eventually, the error between the actual and expected output is
minimized
• In this way, the output of the current iteration becomes the input and affects the next output. This
is repeated until the correct output is produced. The weights at the end of the process would be
the ones on which the neural network works correctly
 Backpropagation
• https://www.youtube.com/watch?v=fQ376G-Ek1E