Task 1: Reflective account on machine learning algorithms and open source

libraries

1. Scikit-learn
Scikit-learn is a software machine learning library for the Python programming language.
Scikit-learn was initially developed by David Cournapeau as a Google summer of code
project in 2007.Later Matthieu Brucher joined the project and started to use it as part of
his thesis work but now the project now has more than 30 active contributors.
Scikit-learn is an open source machine learning library that supports supervised and
unsupervised learning. It also provides various tools for model fitting, data pre-
processing, model selection and evaluation, and many other utilities. It also provides
dozens of built-in machine learning algorithms and models, called estimators. Each
estimator can be fitted to some data using its fit method.
The best aspect about this framework is the availability of well integrated algorithms
within the Python development environment. It is quite easy to install within most Python
IDEs and relatively easy to use as well. The software provides tutorials that are accessible
online which ensures understanding this library quite easy allowing one to become
proficient in machine learning.
scikit-learn allows seamless integration between math-based and data based libraries e.g.
the ability to use numpy, Sympy, Matplotlib, SciPy and pandas frameworks which
removes the need to extra data transformations.
One good thing with the Scikit learn is that, with just few lines of code, one can develop a
simple machine learning model with many features like model validation, data splitting
for training/testing and many others.
For a good start refer to:
 Quick Start Tutorial http://scikit-
learn.org/stable/tutorial/basic/tutorial.html
 User Guide http://scikit-
learn.org/stable/user_guide.html
 API Reference http://scikit-
learn.org/stable/modules/classes.html
 Example Gallery http://scikit-
learn.org/stable/auto_examples/index.html
 2. CNTK
CNTK is a powerful computation-graph based deep-learning toolkit for training and
evaluating deep neural networks developed by Microsoft and it is a part of the ONNX
format initiative, which allows easy conversion of models between different neural
toolkit frameworks.
It enables us to create some popular deep learning systems like feed-forward neural
network time series prediction systems and Convolutional neural network (CNN) image
classifiers
CNTK has highly optimised built-in components that can handle multi-dimensional
dense or sparse data from Python, C++ or BrainScript. Also CNTK can be used to
implement Reinforcement learning, Generative Adversarial Networks (GANs),
Supervised as well as Unsupervised learning. This makes it a high-level deep learning
frame-work.
Refernces
 Limer, E. (2013, October 1). Heck yes! The first free wireless plan is finally here. Retrieved from
https://gizmodo.com/heck-yes-the-first-free-wireless-plan-is-finally-here
3. Theano
Theano is an open source project primarily developed by a machine learning group at
Universite De Montreal for mathematical computations. It has been around since 2008
and has been used widely in development of some of the most complex model
architectures for neural networks. It allows the user to define optimize and evaluate
mathematical expressions involving multi-dimensional arrays efficiently using Python.
Theano has many features which makes it valuable to some deep-learning learners. These
features include;

 Strong integration with NumPy

 While using a GPU, it performs data intensive computations much faster than on
a CPU.
 Best in speed and stability optimization.
 Theano generates dynamic C code which helps in evaluating expressions faster.
 Theano also implements extensive unit testing and self-verification which results
in detection and diagnosis of many types of errors
 equipped to perform large-scale computationally intensive scientific investigation
  it handles mathematical expressions very efficiently including the ones which have
multi-dimensional arrays. 

Before using this open-source framework, one should be able to write at least simple
neural networks in Python and train it with gradient Descent or backpropagation
theano Relies on the mathematical side of deep learning more often and has lower
number of followers today even though it was the first set of libraries launched. It does
not provide you with pre-built models that you can train on your data set, instead it
provides you with tools to build your own machine learning models.
Reference
 recode Minds pvt Ltd, January 5th 2020, ‘beginners guide to theano for deep
learning’

4. KERAS
Keras is a powerful and easy-to-use free open source Python library for developing and
evaluating deep learning models. It wraps the efficient numerical computation libraries
Theano and Tensor Flow and allows you to define and train neural network models in
just a few lines of code. Keras offers simple and consistent high-level APIs and
follows best practices to reduce the cognitive load for the users. Keras is written in
python which makes it easy to program and also to debug.Keras offers a wide range of
features eg, Consistent, simple and extensible API, easy to achieve the result without any
frills, multiple platform support, user friendly framework which runs on both CPU and
GPU and a Highly scalability of computation.
These features make Keras a choice for many neural network model development. Keras
contains numerous implementations of commonly used neural-network building blocks
such as layers, objectives, activation functions, optimizers, and a host of tools to make
working with image and text data easier to simplify the coding necessary for writing deep
neural network code. Additionally, Keras has support for convolutional and recurrent
neural networks. It supports other common utility layers like dropout, batch
normalization, and pooling.
Keras is a preferred deep leraning library since it enables users to build and train models
efficiently, clean and easy to understand arrangement of layers, model training is easy
requiring only data, and perfect work with few number of codes thus saving time.
Keras was developed and maintained by François Chollet, a Google engineer to ensure
Modularity, Minimalism, Extensibility and use of Python.

Reference

 LeCun, Yann, Corinna Cortes, and C. J. Burges. MNIST handwritten digit

database. AT&T Labs [Online]. Available: http://yann. lecun. com/exdb/mnist 2
(2010).
 https://www.javatpoint.com/keras

5. Tensorflow
TensorFlow is a Python-friendly open source library for numerical computation that
helps to makes machine learning faster and easier.. It has a comprehensive, flexible
ecosystem of tools, libraries, and community resources that lets researchers push the
state-of-the-art in ML, and gives developers the ability to easily build and deploy ML-
powered applications.
Tensors offers integration with many other high-level APIs such us Keras, 4

TensorFlow provides a collection of workflows with intuitive, high-level APIs for

developers to create machine learning models in numerous languages. Developers have
the option to deploy models on a number of platforms such as on servers, in the cloud, on
mobile and edge devices, in browsers, and on many other JavaScript platforms. This
enables developers to go from model building and training to deployment much more
easily.
TensorFlow offers a wide range of amazing features that makes more developers prefer
it. Tensorflow is easy to run, fast debugging, effective, scalable, abstraction1, easy
experimentation and flexible. It also provides a number of tutorials to get everyone
started quiet easily and catch up with the flow. Also there are easily integrable pre-trained
model layers which help in time saving.
TensorFlow has a large number of users (developers) making it have a vast number of
resources (models) which makes model developing and training quiet easy.

references
 https://www.javatpoint.com/tensorflow
Second, you are to review (250 to 300 words) the following machine learning
algorithms, indicating their most well-known uses.

a) Support Vector Machines

 Int: Support vector machines (SVMs) are powerful yet flexible supervised
machine learning algorithms which are used both for classification and regression.
SVMs are extremely popular because of their ability to handle multiple continuous
and categorical variables. The goal of the SVM algorithm is to create the best line
or decision boundary that can classify n-dimensional space into classes so that we
can easily put the new data point in the correct category in the future. This best
decision boundary is called a hyperplane. SVM chooses the extreme points/vectors
that help in creating the hyperplane (best boundary line with minimum errors).
These extreme cases are called as support vectors, and hence algorithm is termed
as Support Vector Machine.
 SVM algorithm can be used for Face detection, image classification, text
categorization
 SVM can be of two types:
 Linear SVM for linearly separable data, or data that can be
distinguished by a straight boundary line.
 non-linear SVM for non-linearly separable data. Data which cannot
be separated by a straight line.
 SVM classifies data in two main steps:
 First, SVM will generate hyperplanes iteratively that segregates the
classes in best way.
 Then, it will choose the hyperplane that separates the classes
correctly ensuring that errors are reduced and distance betrween the
two classes is maximized.
 Though SVM is quite good for classification of datasets, it cannot be used for
large datasets without a clear margin of separation, ie when data is overlapping. In
cases where the number of features for each data point exceeds the number of
training data samples, the SVM will underperform.
 SVM ia actively being used in speech recognition, text classification, facial
expression classification, texture classification.
References
 https://www.tutorialspoint.com/machine_learning_with_python/classification_algorithm
s_support_vector_machine.htm
 

b) Maximum Entropy
 The Max Entropy classifier is a discriminative classifier commonly used in
Natural Language Processing, Speech and Information Retrieval problems. Its best
when implemented in python due to the high number of numerical computations
that one has to compute for the input weights estimation.
 Entropy is a measure of the randomness in the information being processed. The
higher the entropy, the harder it is to draw any conclusions from that information.
So maximum entropy algorithm is based on the principle of maximum entropy and
works to ensure minimum assumptions.
 The Max Entropy classifier can be used to solve a large variety of text
classification problems such as language detection, topic classification, sentiment
analysis and more.
 Due to the minimum assumptions that the Maximum Entropy classifier makes, we
regularly use it when we don’t know anything about the prior distributions and
when it is unsafe to make any such assumptions.
 The Max Entropy requires more time to train. Primarily due to the optimization
problem that needs to be solved in order to estimate the parameters of the model.
Nevertheless, after computing these parameters, the method provides robust results
and it is competitive in terms of CPU and memory consumption.
 One main limitation is the possibility of over-fitting, limiting the capacity of the
model to generalize well to independent data.
 In the real world, maximum entropy is being applied in,

Refernces
 Limer, E. (2013, October 1). Heck yes! The first free wireless plan is finally here. Retrieved from
https://gizmodo.com/heck-yes-the-first-free-wireless-plan-is-finally-here

c) Recurrent Neural Networks

 Recurrent Neural Networks are algorithms that are designed to interpret temporal
or sequential information. RNNs use other data points in a sequence to make better
 predictions. They do this by taking in input and reusing the activations of previous
nodes or later nodes in the sequence to influence the output.
 RNNs have internal memory that help them remember important details such as
inputs received. This feature which makes them be very precise in predicting what
is coming next. Hence, they are the most preferred algorithm for sequential data
like time series, speech, text, audio, video, sequence prediction, predicting a single
output.
 RNN work by performing iterations in such a way that the output from the
previous step is used as the input for the current step. This algorithm does this
because the previous output has crucial information about what is coming next and
thus prediction in a sequence is simple using the Recurrent Neural Network. This
is most important feature in the RNN that makes the algorithm unique from other
algorithms.
 Recurrent neural network are even used with convolutional layers to extend the
effective pixel neighborhood.
 Limiting factor to RNN is found in training the model. Its time consuming and
difficult task. Also a RNN cannot process very long sequences if using tanh or
relu as an activation functions

Reference … tutorials point, javatpoint, upgradblog, geeksforgeeks

d) Multinomial Naïve Bayes

 Multinomial Naive Bayes algorithm is a probabilistic learning method that is
mostly used in Natural Language Processing (NLP). The algorithm is based on the
Bayes theorem and predicts the tag of a text such as a piece of email or newspaper
article. It calculates the probability of each tag for a given sample and then gives
the tag with the highest probability as output
 Bayes theorem, formulated by Thomas Bayes, calculates the probability of an
event occurring based on the prior knowledge of conditions related to an event
This formula is used to calculate the probabilities: P(A|B) = P(A) * P(B|A)/P(B)
 Multinomial Naïve Bayes algorithm is easy to implement since we only have to
calculate the probabilities. The algorithm can be used for both continuous and
descrete data, and can also handle large data sets.
 A real life application of this algorithm is in Face recognition, Weather prediction,
Medical diagnosis, Spam detection, Age/gender identification, Language
identification, Sentimental analysis, News classification
 Though easy to implememnt, the Naive bayes algorithm’s accuracy is lower
compared to other probability algorithms, also its only used to textual data
classification and not numerical data
 references

 https://www.tutorialspoint.com/machine_learning_with_python/classification
_algorithms_naive_bayes.html


e) Convolutional Neural Networks

 A Convolutional Neural Network is a Deep Learning algorithm which can take in
an input image, assign weights to various aspects in the image so as to be able to
differentiate one from the other.
 Once they become differentiable, using various activation functions the
Convolutional Neural Network Model can perform several tasks in the Image
Processing domain including Image Recognition, Image Classification, Object and
Face Detection
 A Convolution Neural Network Model consists of several layers through which
the input images undergo pre-processing to get the output. Ie OUTPUT LAYER
which allows input of images into the model for processing, CONVOLUTION
LAYER where mathematical operations of convolution are performed, POOLING
LAYER whose aim is to reduce the size of feature map to reduce computational
cost, FULLY CONNECTED LAYER where activation functions are applied to
the inputs to predict or label the input, and lastly the OUTPUT LAYER which
gives the computational results of the model.
 The CNN are best in visual image data processing, but have a small disadvantage
in that they do not work well with a sequence of images (videos) and fail in
interpreting the temporal information & blocks of text.
 In real life occurrence Convolution Neural Networks are applied in Decoding
Facial Recognition in social media, Analyzing Documents, Understanding
Climate, Object detection for self-driving cars, Image analysis in healthcare,
recommender engines eg in online shopping, The legal organizations, as banking
and insurance, use Optical Character Recognition of handwriting. Among many
others.

Refrences
 Limer, E. (2013, October 1). Heck yes! The first free wireless plan is finally here. Retrieved from
https://gizmodo.com/heck-yes-the-first-free-wireless-plan-is-finally-here
 f) Logistic Regression
 Logistic regression is one of the most popular Machine Learning algorithms used
for solving the classification problems, which comes under the Supervised
Learning technique. It is used for predicting the categorical dependent variable
using a given set of independent variables.
 Logistic regression predicts the output of a categorical dependent variable.
Therefore the outcome must be a categorical or discrete value. It can be either Yes
or No, 0 or 1, true or False, etc. but instead of giving the exact value as 0 and 1, it
gives the probabilistic values which lie between 0 and 1
 In Logistic regression, instead of fitting a regression line, we fit an "S" shaped
logistic function, which predicts two maximum values (0 or 1).
 Logistic Regression is a significant machine learning algorithm because it has the
ability to provide probabilities and classify new data using continuous and discrete
datasets.
 Logistic Regression can be classified into three types: Binomial: where there can
only be two types of dependent variables eg 0 and 1, Multinomial where there can
be 3 or more unordered types of dependent variables eg cat, dog, house, etc and
Ordinal where there can be 3 or more ordered types of dependent variables eg
high, low, medium.
 The algorithm makes some assumptions which are The dependent variable must be
categorical in nature and The independent variable should not have multi-
collinearity.
 In real life Logistic regression is mainly applied in medicine when studying
subsequent molecules, text editing eg toxic speech detection, hotel booking where
machine try to predict the next intentions of the user, making recommendation in
gaming applications.

References
 https://www.tutorialspoint.com/machine_learning_with_python/classification_algorithms_logis
tic_regression.htm
